When to use a Worker?

[yvi]

Can any of you more experienced people tell me roughly when it is a good idea to write a TheSchwartz-scheduled Worker? Looking at the workers in existence, it's for things that need to work on all users or all clusters (latest feed, invite code distribution, userpic renaming), things that can take a long time (importer) or that don't need to happen instantly and may need to be tried a few times (crossposting).

I am asking both for curiosity and because I am working on merging tags (taking a list of tags and for each entry with that tag, adding the new tag is added and deleting the old tags), which I think is not such a heavy load that it would require a worker, but I thought I'd ask.

Comments

[denise]

The impression I get is that it's for things that can be done asynchronously -- that don't need to be completed immediately in order for the workflow the user is doing, or that might take longer than the 5-10 seconds a user is generally willing to wait without reloading a page.

From a user experience standpoint, and completely ignoring the technical and of things, anything that might take more than a few seconds is going to lead to people thinking that something's not right and hitting Refresh to try again, which will result in multiple jobs, etc. As far as I'm concerned, anything that might take more than 5-10 seconds on an average account (ie, not someone like scans_daily with their 5000 tags) should be done asynchronously with the user notifed when it's done...

[pauamma]

What you said is essentially correct. (You forgot things that happen periodically without human intervention, like expiring paid accounts or renewing them if said to recurring billing by credit card, but that's not relevant here.)

But in this case, I don't think it's needed. Merging a tag isn't really different, here, from deleting a tag (not just removing it from an entry - completely deleting it) or an access group, neither of which (IIRC) uses TheSchwartz.

[yvi]

Neither deleting nor renaming tags use it, no. So I think I am in the clear then :) Thanks!

[mark]

There are also different kinds of workers... Gearman and TheSchwartz. They are used in different scenarios.

Gearman is typically used when we have things we need done quickly, the user is waiting. I.e., the search system uses a Gearman worker to send out the search job to Sphinx. The directory search also uses Gearman. The image resizer uses it. "Do this right now" kind of tasks.

TheSchwartz is for things that must be done, but they could take a while and we don't care if they're done right now. "Do this sometime, just make sure it runs" things.