badass django with celery 🚬

In this article we will see how to schedule periodic task, like sending mail to managers every 2 hours or sending mail at 4 am daily

Before getting started let us understand some of Jargons that you’ll hear in this article, for detailed information please do Google search on them

Celery : We will use this to execute or asyncronous tasks

Celery Beat : Celery beat is used to schedule our task

RabbitMQ : This is used as message broker (in simple terms — forms links) to send task to celery

Redis : This works same as RabbitMQ, another message broker

Cronjob : Cron is a utility that schedule scripts or commands to be run automatically at specified time and date or intervals and these tasks or jobs are what we call as cron job

Crontab : It is a list of commands that you want to run on a regular schedule, mostly we will be using this to setup our task to run at every x interval, for detail on this follow this link of Crontab

Note — We can run scheduled task through either with cronjob or celery, but Celery gives are more control and that’s why it is mostly preferred with django

Now enough of talking lets start scheduling our task in django

Step 1 : Install Celery by running below command, and don’t forget to add that in requirements.txt (optional)

to verify installation of celery run command

it will show you output like this, if you get ouput same as below, you can move to step 2

Step 2 : Create a file name as celery.py alongside with the settings.py and urls.py files as illustrated below, if you have created setting package then put this file inside settings package

now add the code below in celery.py file, read comments for better understanding

Now we have to edit the init file shown below

Add the code below in init file

Step 3 : Now we will add code which will be executed by our scheduler, create a file name as tasks.py shown below

Now add the code which you want scheduler to run, i have added the sample code below, our sample code update first user in database

Step 4 : Now we will create schedule for this task to run at every 10 seconds, So now create a file name as celery_config.py file as show in below directory

add code below in celery_config.py file

and don’t forget to import this file in your settings.py file as shown below

Step 5 : Here we will use redis as our message broker, which will be responsible for sending tasks to celery at periodic interval

Download redis from its offical page => Download

after downloading fire up server by opening terminal and execute

You can test that Redis is working properly by typing this into your terminal

Redis should reply with PONG , You also need to add Redis as a dependency in the Django Project (optional), after doing this add the following setting to your setttings.py file

Step 6: Now finally we are done, we need to test our scheduler by running command below, run below commands in 2 separate terminal

here,

  • -A denote for app name which is added next to it in my case it is my_app, change accordingly your app name
  • worker is what we want to run
  • -l denote level of log we want, we are keeping this as info

In next terminal run

change my_app to your app name, this command is used to beat schedule at specific time defined in celery_config.py

Finally if you didn’t get any error you should be able to see username changing for user with primary_key 1, every 10 seconds 🎉🎉🎉

Conclusion : Setting up celery is bit tricky make sure, you cross check settings twice after implementing steps from this article, and don’t add files in wrong directory, then you should be able to execute schedule task properly

Check out my article on :

SonarQube with pre commit hook ⚓

--

--

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store