8

I want to schedule and run a lot of jobs in the background during a web application execution.
The web app is built on top of Symfony 2 and Doctrine 2.

I know the job-processing can be done with libraries like Resque or Sidekiq. However, these libraries and my application are written in different languages, so I am wondering how I can run Sidekiq jobs written in Ruby which should integrate with my app written in PHP.

What I'm asking myself is if the only way to do this is rewriting a large amount of code to query the database from PHP to ruby, to be able to execute the job in Sidekiq/Resque.

5
  • have you considered php solutions ?
    – Calimero
    Jan 23, 2014 at 15:54
  • did you take a look on github.com/chrisboulton/php-resque ? Jan 23, 2014 at 15:55
  • I considered running a PHP script with a simple endless loop in a cmd but I think it's not the most appropriate solution. @Piotr I will have a look at the project you linked me, it seems interesting but I'm wondering about the performances
    – Stefano
    Jan 23, 2014 at 15:55
  • It depends on how big is your project. If it's not huge then you can use endless loop and read queue from DB, I know one in-house solution like that and it's working perfectly fine even for big companies. If you don't need queue priority, multiple workers etc "endless loop" and watchdog that checks if endless loop is still working should be fine Jan 23, 2014 at 16:00
  • my application is not as big as facebook, but what scares me is the fact that there will be a lot of queries to the database that will have to be executed every minute, then I will have to send notifications to browsers and emails, so the jobs must be processed reliably and fast
    – Stefano
    Jan 23, 2014 at 16:02

2 Answers 2

6

Have you taken a look at Gearman? It lets you run background jobs just like Sidekiq, but it's language agnostic. For example, you can use PHP for everything, or you can queue up jobs in PHP and have the actual workers written in Ruby.

1
  • Queuing is a fundamental component of Gearman, so yes to that. Scheduling, however, is not supported by Gearman. You can easily make your own though. Create a cron job or periodic task that runs every X minutes, it checks a database table containing a timestamp column, job name, and serialized arguments, and runs any jobs whose timestamps are before the current time.
    – Michael S.
    Jan 24, 2014 at 17:55
6

I have made use of Resque in several projects using https://github.com/chrisboulton/php-resque and https://github.com/chrisboulton/php-resque-scheduler

Its been working really well, I even made a Symfony bundle to make working with it really easy. https://github.com/mcfedr/resque-bundle - supports background jobs and scheduled jobs. Much more powerful than using cron.

The main reason for choosing Resque over other options is that it works on Redis, which is easy to deploy and scale. On AWS I use Elasticache managed instances for a completely hassle free setup.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.