The key is to use the "mins" parameter in the script and setup the script as a cron.
The "mins" parameter searches for all data ingested into the system in the last X minutes.
Say you set mins=10, the script will search and download all data ingested in the last 10 mins.
You can turn this script into a cron thereby obtaining a new feed of data every 10 mins. Of course, you can run this every 60 minutes, every day, or every week and set the mins parameter accordingly.
