Data Science Quick Tip #002: Running a Cronjob from Within a Flask API!
Hey there folks! Back at it again with another quick tip for you all. I got a lot of very kind and positive remarks about the last post, so I’m happy to be back at it again this week!
Today, we’re going to be talking about how to run a cronjob from within a Flask API. Now if you’re not familiar with Flask and its relevance for machine learning, I’d suggest you check out any of the multitude of fantastic posts that cover that more thorough. Actually, let me make it easy on you and just suggest this one here. There is an endless source of great articles on Medium just like that one linked above.
As helpful as those articles are, my goal with these posts is to hopefully teach you something you might not see elsewhere. So while there are a million articles about the usefulness of Flask for machine learning deployment, I personally haven’t seen any that cover how to run a basic cronjob from directly within a Flask API. Once you see how to do this, you’ll see just how simple and quick it is to implement. Before we jump into how to do this, let’s briefly cover why one might want to do this.
Why would anybody want to run a cronjob from within a Flask API?
I hope it goes without saying that if I’m writing a post on how to do something, it means I had a reason to figure it out for myself in the first place! In my scenario, I enhanced data coming into an API with additional data loaded in memory as a Pandas DataFrame. That additional data came in the form of a static CSV file that was in a place my Flask API could readily access, and it was updated twice a week via external processes. In order to make sure that enhancing data was being appropriately refreshed, I needed a means to run a simple cronjob without bringing down the core functionality of the Flask API.
At the start of my journey, I toyed around with several ideas. I initially wondered if I would simply have to bring down the whole Flask API deployment and re-load it every time the enhancing data was refreshed, but that clearly wasn’t an optimal choice. Another idea was to create a separate deployment just for refreshing the enhancing data and have my Flask API somehow reference that, but that also seemed unnecessarily complex.
Fortunately, I discovered what I’m about to share with you, and not only did it do the job for me, but it’s also super simple to implement!
How do I run a simple cronjob from within a Flask API?
Alrighty, so now that we’ve covered the “why,” let’s turn our attention to the “how.” Perhaps one of the reasons I love Python so much is that there are so many helpful libraries out there just ripe for the picking. In our case here, we’re going to use a library called APScheduler. APScheduler actually has a lot more functionality than what we’re going to make use of, but we’re going to focus specifically on the BackgroundScheduler.
(Oh, before going further, here’s the link to my supporting GitHub repo for this post. I’ll be sure to link it at the bottom of the post, too.)
Just as the name sounds, the BackgroundScheduler starts a very small thread behind the scenes specifically for running the cronjob. After properly pip installing the library, importing it into your Flask script as as simple as this.
Believe it or not, we’re already a third of the way to being done. Now that we’ve imported the BackgroundScheduler, we need to instantiate it as an object and then issue the command to properly start it up. As the screenshot below will show, this is very simple to do. In my case, we’ll name this instance of the BackgroundScheduler as “sched.”
Two thirds down, one to go. Now the thing that I love about APScheduler is its ability to simply add a decorator to a function in order for it to apply the cron functionality to it. In the screenshot below, I’m adding a cronjob decorator that will run every minute. As soon as that second hand hits the 12, a very simple “Hello world!” message will be printed directly to the console. Obviously this is a very simple case, so if you want to see all the options of what you can alter within the cron decorator, check out this documentation here.
Wouldn’t you know it, we’re already done! When I go and start up my Flask API, “Hello world!” gets printed out directly to the console every minute just like in the screenshot below.
Easy peasy! The cronjob runs when it’s supposed to while the core functionality of the Flask API remains intact.
And that’s it for this post! If you’d like to delve further into my code, be sure to check out my GitHub repo. It contains the precise instructions on how to properly install the libraries you’ll need as well as how to properly invoke my little Python script there. Hope you found this quick tip to be helpful! If there’s anything else you’re curious about, share your ideas with me as they might end up in a future post. Thanks again for reading!