diff --git a/Add-on Documentation/Data Processing/Worker.md b/Add-on Documentation/Data Processing/Worker.md deleted file mode 100644 index 32c0c51..0000000 --- a/Add-on Documentation/Data Processing/Worker.md +++ /dev/null @@ -1,151 +0,0 @@ -# Worker Add-on - -Workers are long running background processes. They are typically used for -anything from sending emails to running heavy calculations or rebuilding caches -in the background. - -Each worker started via the Worker add-on runs in a seperate isolated -container. The containers have exactly the same runtime environment defined by -the stack chosen and the buildpack used and have the same access to all of the -deployments add-ons. - -Note: Workers sometimes get interrupted and restarted on a different host for -the following reasons: - - single instances can run into issues and need to be replaced - - containers are redistributed to provide the best performance - - security updates are applied - -This means all your worker operations should be idempotent. If possible a `SIGTERM` -signal is send to your worker before the shutdown. - -## Adding the Worker Add-on - -Before you can start a worker, add the add-on with the addon.add command. - -~~~ -$ cctrlapp APP_NAME/DEP_NAME addon.add worker.single -~~~ - -## Starting a Worker - -Workers can be started via the command line client's worker.add command. - -For the Luigi stack (only supporting PHP), use the PHP filename as the `WORKER_NAME`. But for apps on the Pinky stack, first specifiy how to start the worker by adding a new line to your app's `Procfile` and then use that as the `WORKER_NAME`. - -~~~ -$ cctrlapp APP_NAME/DEP_NAME worker.add WORKER_NAME [WORKER_PARAMS] -~~~ - -Enclose multiple WORKER_PARAMS in double quotes. - -~~~ -$ cctrlapp APP_NAME/DEP_NAME worker.add WORKER_NAME "PARAM1 PARAM2 PARAM3" -~~~ - -## List Running Workers - -To get a list of currently running workers use the worker command. - -~~~ -$ cctrlapp APP_NAME/DEP_NAME worker -Workers - nr. wrk_id - 1 WRK_ID -~~~ - -You can also get all the worker details by appending the WRK_ID to the worker command. - -~~~ -$ cctrlapp APP_NAME/DEP_NAME worker WRK_ID -Worker -wrk_id : WRK_ID -command : WORKER_NAME -params : "PARAM1 PARAM2 PARAM3" -~~~ - -## Stopping Workers - -Workers can be either stopped via the command line client or by exiting the process with a zero exit code. - -### Via Command Line - -To stop a running worker via the command line use the worker.remove command. - -~~~ -$ cctrlapp APP_NAME/DEP_NAME worker.remove WRK_ID -~~~ - -To get the WRK_ID refer to the listing workers section above. - -### Via Exit Codes - -To stop a worker programatically use UNIX style exit codes. There are three distinct exit codes available. - - * exit (0); // Everything OK. Worker will be stopped. - * exit (1); // Error. Worker will be restarted. - * exit (2); // Error. Worker will be stopped. - -For more details refer to the [PHP example](#php-worker-example) below. - -## Worker log - -As already explained in the [Logging section](https://www.cloudcontrol.com/dev-center/platform-documentation#logging) all stdout and stderr output of workers is redirected to the worker log. To see the output in a tail -f like fashion use the log command. - -~~~ -$ cctrlapp APP_NAME/DEP_NAME log worker -[Fri Dec 17 13:39:41 2010] WRK_ID Started Worker (command: 'WORKER_NAME', parameter: 'PARAM1 PARAM2 PARAM3') -[Fri Dec 17 13:39:42 2010] WRK_ID Hello PARAM1 PARAM2 PARAM3 -[...] -~~~ - -## Removing the Worker Add-on - -To remove the Worker add-on use the addon.remove command. - -~~~ -$ cctrlapp APP_NAME/DEP_NAME addon.remove worker.single -~~~ - -## PHP Worker Example - -The following example shows how to use the exit codes to restart or stop a worker. - -~~~php -// read exit code parameter -$exitCode = isset($argv[1]) && (int)$argv[1] > 0 ? (int)$argv[1] : 0; -$steps = 5; - -$counter = 1; -while(true) { - print "step: " . ($counter) . PHP_EOL; - if($counter == $steps){ - if($exitCode == 0) { - print "All O.K. Exiting." . PHP_EOL; - } else if ($exitCode == 2){ - print "An error occured. Exiting." . PHP_EOL; - } else { - print "An error occured. Restarting." . PHP_EOL; - } - print "Exitcode: " . $exitCode . PHP_EOL . PHP_EOL; - exit($exitCode); - } - sleep(1); - $counter++; -} -~~~ - -Running this worker with the exit code set to 2 would result in the following output and the worker stopping itself. - -~~~ -$ cctrlapp APP_NAME/DEP_NAME worker.add WORKER_NAME 2 -$ cctrlapp APP_NAME/DEP_NAME log worker -[Tue Apr 12 09:15:54 2011] WRK_ID Started Worker (command: 'WORKER_NAME', parameter: '2') -[Tue Apr 12 09:15:54 2011] WRK_ID step: 1 -[Tue Apr 12 09:15:55 2011] WRK_ID step: 2 -[Tue Apr 12 09:15:56 2011] WRK_ID step: 3 -[Tue Apr 12 09:15:57 2011] WRK_ID step: 4 -[Tue Apr 12 09:15:58 2011] WRK_ID step: 5 -[Tue Apr 12 09:15:58 2011] WRK_ID An error occured. Exiting. -[...] -~~~ - diff --git a/Add-on Documentation/Deployment/Cron.md b/Add-on Documentation/Deployment/Cron.md index bd1d551..6a4639c 100644 --- a/Add-on Documentation/Deployment/Cron.md +++ b/Add-on Documentation/Deployment/Cron.md @@ -18,8 +18,8 @@ Cronjobs are regular requests against your app and are subject to the same 120s timelimit. If you need more control over when and how often tasks are run and/or have -tasks that take longer than 120 seconds we recommend using the -[Worker](https://www.cloudcontrol.com/add-ons/worker) or +tasks that take longer than 120 seconds we recommend using a +[Worker](https://www.cloudcontrol.com/dev-center/platform-documentation#workers) or [IronWorker](https://www.cloudcontrol.com/add-ons/iron_worker) Add-on. There are also the [IronMQ](https://www.cloudcontrol.com/add-ons/iron_mq) and the [CloudAMQP](https://www.cloudcontrol.com/add-ons/cloudamqp) message queues @@ -107,4 +107,3 @@ $ cctrlapp APP_NAME/DEP_NAME addon.remove cron.OPTION ~~~ Please note: Removing the Add-on will not automatically remove all Cron jobs. - diff --git a/Guides/Python/Celery.md b/Guides/Python/Celery.md index 47ce4db..2d38729 100644 --- a/Guides/Python/Celery.md +++ b/Guides/Python/Celery.md @@ -1,7 +1,7 @@ # Deploying Celery on cloudControl [Celery] is an asynchronous task queue/job queue based on distributed message passing. It is focused on real-time operation, but supports scheduling as well. -In this tutorial we're going to show you how to deploy an example Celery app using the [CloudAMQP Add-on] and the [Worker Add-on] on [cloudControl]. +In this tutorial we're going to show you how to deploy an example Celery app using the [CloudAMQP Add-on] and a [Worker] on [cloudControl]. ## The Example App Explained First, lets clone the example code from Github. It is based on the official [first steps with Celery guide][celeryguide] and also includes [Flower] the Celery web interface. @@ -67,12 +67,6 @@ As we chose to use AMQP as a broker, we add the CloudAMQP Add-on now. $ cctrlapp APP_NAME/default addon.add cloudamqp.lemur ~~~ -We also need to add the Worker Add-on to be able to start the workers later. - -~~~bash -$ cctrlapp APP_NAME/default addon.add worker.single -~~~ - Since we are reading the AMQP URL for the broker from the environment in both, the `Procfile` and the Python code we have to enable providing Add-on credentials as environment variables which is disabled per default for Python apps. We also set another environment variable called `FLOWER_AUTH_EMAIL` that is passed to the Flower web process for authentication purposes. Without this, the web interface would be public showing your secret AMQP credentials and allowing people to stop your workers. @@ -96,7 +90,7 @@ Delta compression using up to 4 threads. Compressing objects: 100% (4/4), done. Writing objects: 100% (6/6), 605 bytes, done. Total 6 (delta 0), reused 0 (delta 0) - + -----> Receiving push -----> Preparing Python interpreter (2.7.2) -----> Creating Virtualenv version 1.7.2 @@ -107,14 +101,14 @@ Total 6 (delta 0), reused 0 (delta 0) Running virtualenv with interpreter /usr/bin/python2.7 -----> Activating virtualenv -----> Installing dependencies using pip version 1.2.1 - + [...] - + Successfully installed celery flower billiard python-dateutil kombu tornado anyjson amqp Cleaning up... -----> Building image -----> Uploading image (4.3M) - + To ssh://APP_NAME@cloudcontrolled.com/repository.git * [new branch] master -> master ~~~ @@ -122,7 +116,7 @@ To ssh://APP_NAME@cloudcontrolled.com/repository.git Last but not least deploy the latest version of the app with the cctrlapp deploy command. ~~~bash -$ cctrlapp APP_NAME/default deploy +$ cctrlapp APP_NAME/default deploy ~~~ At this point you can see web interface at `http://APP_NAME.cloudcontrolled.com`. But it hasn't got any workers yet. @@ -139,7 +133,7 @@ $ cctrlapp APP_NAME/default worker # and also check the worker's log output with $ cctrlapp APP_NAME/default log worker [TIMESTAMP] WRK_ID Started worker (command: 'celery -A tasks worker --loglevel=info ', parameter: '') -[TIMESTAMP] WRK_ID +[TIMESTAMP] WRK_ID [TIMESTAMP] WRK_ID -------------- celery@HOSTNAME v3.0.15 (Chiastic Slide) [TIMESTAMP] WRK_ID ---- **** ----- [TIMESTAMP] WRK_ID --- * *** * -- [Configuration] @@ -151,7 +145,7 @@ $ cctrlapp APP_NAME/default log worker [TIMESTAMP] WRK_ID - *** --- * --- [Queues] [TIMESTAMP] WRK_ID -- ******* ---- . celery: exchange:celery(direct) binding:celery [TIMESTAMP] WRK_ID --- ***** ----- -[TIMESTAMP] WRK_ID +[TIMESTAMP] WRK_ID [TIMESTAMP] WRK_ID [Tasks] [TIMESTAMP] WRK_ID . tasks.add [TIMESTAMP] WRK_ID [TIMESTAMP: WARNING/MainProcess] celery@HOSTNAME ready. @@ -210,10 +204,9 @@ This guide showed how to run both Flower aswell as a Celery worker on cloudContr [Celery]: http://celeryproject.org/ [CloudAMQP Add-on]: https://www.cloudcontrol.com/add-ons/cloudamqp -[Worker Add-on]: https://www.cloudcontrol.com/add-ons/worker +[Worker]: https://www.cloudcontrol.com/dev-center/platform-documentation#workers [cloudControl]: http://www.cloudcontrol.com [celeryguide]: http://docs.celeryproject.org/en/latest/getting-started/first-steps-with-celery.html [Flower]: http://docs.celeryproject.org/en/latest/userguide/monitoring.html#flower-real-time-celery-web-monitor [Python buildpack]: https://github.com/cloudControl/buildpack-python [Procfile]: https://www.cloudcontrol.com/dev-center/platform-documentation#buildpacks-and-the-procfile - diff --git a/Platform Documentation.md b/Platform Documentation.md index 27dc4ba..e937277 100644 --- a/Platform Documentation.md +++ b/Platform Documentation.md @@ -774,7 +774,7 @@ Please note that Secure WebSockets connections can only be established using `*. **TL;DR:** * Web requests are subject to a time limit of 120s. - * Scheduled jobs are supported through different Add-ons. + * Scheduled jobs are supported through the Cron Add-on or workers. * Background workers are the recommended way of handling long running or asynchronous tasks. Since a web request taking longer than 120s is killed by the routing tier, longer running tasks have to be handled asyncronously. @@ -785,7 +785,149 @@ For tasks that are guaranteed to finish within the time limit, the [Cron add-on] ### Workers -Tasks that will take longer than 120s to execute, or that are triggered by a user request and should be handled asyncronously to not keep the user waiting, are best handled by the [Worker add-on]. Workers are long-running processes started in containers. Just like the web processes but they are not listening on any port and therefore do not receive http requests. You can use workers, for example, to poll a queue and execute tasks in the background or handle long-running periodical calculations. More details on usage scenarios and available queuing Add-ons are available as part of the [Worker Add-on documentation]. +Workers are long-running processes started in containers just as the web +processes. The difference between web processes and workers is, that they are +not listening on any port and therefore do not receive http requests. Workers +exploits it's full potential in handling tasks that will take longer than 120s +to execute, or that are triggered by a user request which are handled +asynchronously to prevent idle time for users. For example, handling +long-running periodical calculations or to poll a queue and execute tasks in the +background. + +You can create multiple workers for individual tasks. Each worker runs in a +separate isolated container. The containers have exactly the same runtime +environment defined by the stack chosen and the buildpack used and have the same +access to all of the deployments add-ons. + +Note: Workers sometimes get interrupted and restarted on a different host for +the following reasons: + - single instances can run into issues and need to be replaced + - containers are redistributed to provide the best performance + - security updates are applied + +This means all your worker operations should be idempotent. If possible a `SIGTERM` +signal is send to your worker before the shutdown. + +#### Starting a Worker + +Depending on the [stack](#stack) your app is running on, you may have to edit the app's +`Procfile` first. +If your app is running on the Pinky stack, add the following line to your app's `Procfile`: + +~~~ +WORKER_NAME: COMMANDLINE_COMMAND_TO_EXECUTE SCRIPT +~~~ + +Workers can be started via the command line client's `worker.add` command. + +~~~ +$ cctrlapp APP_NAME/DEP_NAME worker.add WORKER_NAME [WORKER_PARAMS] +~~~ + +Enclose multiple WORKER_PARAMS in double quotes. + +~~~ +$ cctrlapp APP_NAME/DEP_NAME worker.add WORKER_NAME "PARAM1 PARAM2 PARAM3" +~~~ + +#### List Running Workers + +To get a list of currently running workers use the `worker` command. + +~~~ +$ cctrlapp APP_NAME/DEP_NAME worker +Workers + nr. wrk_id + 1 WRK_ID +~~~ + +You can also get all the worker details by appending the WRK_ID to the worker command. + +~~~ +$ cctrlapp APP_NAME/DEP_NAME worker WRK_ID +Worker +wrk_id : WRK_ID +command : WORKER_NAME +params : "PARAM1 PARAM2 PARAM3" +~~~ + +#### Stopping Workers + +Workers can be either stopped via the command line client or by exiting the process with a zero exit code. + +##### Via Command Line + +To stop a running worker via the command line use the `worker.remove` command. + +~~~ +$ cctrlapp APP_NAME/DEP_NAME worker.remove WRK_ID +~~~ + +To get the WRK_ID refer to the listing workers section above. + +##### Via Exit Codes + +To stop a worker programatically use UNIX style exit codes. There are three distinct exit codes available. + + * exit (0); // Everything OK. Worker will be stopped. + * exit (1); // Error. Worker will be restarted. + * exit (2); // Error. Worker will be stopped. + +For more details refer to the [PHP example](#php-worker-example) below. + +#### Worker log + +As already explained in the [Logging section](https://www.cloudcontrol.com/dev-center/platform-documentation#logging) all stdout and stderr output of workers is redirected to the worker log. To see the output in a tail -f like fashion use the `log` command. + +~~~ +$ cctrlapp APP_NAME/DEP_NAME log worker +[Fri Dec 17 13:39:41 2010] WRK_ID Started Worker (command: 'WORKER_NAME', parameter: 'PARAM1 PARAM2 PARAM3') +[Fri Dec 17 13:39:42 2010] WRK_ID Hello PARAM1 PARAM2 PARAM3 +[...] +~~~ + +#### PHP Worker Example + +The following example shows how to use the exit codes to restart or stop a worker. + +~~~php +// read exit code parameter +$exitCode = isset($argv[1]) && (int)$argv[1] > 0 ? (int)$argv[1] : 0; +$steps = 5; + +$counter = 1; +while(true) { + print "step: " . ($counter) . PHP_EOL; + if($counter == $steps){ + if($exitCode == 0) { + print "All O.K. Exiting." . PHP_EOL; + } else if ($exitCode == 2){ + print "An error occured. Exiting." . PHP_EOL; + } else { + print "An error occured. Restarting." . PHP_EOL; + } + print "Exitcode: " . $exitCode . PHP_EOL . PHP_EOL; + exit($exitCode); + } + sleep(1); + $counter++; +} +~~~ + +Running this worker with the exit code set to 2 would result in the following output and the worker stopping itself. + +~~~ +$ cctrlapp APP_NAME/DEP_NAME worker.add WORKER_NAME 2 +$ cctrlapp APP_NAME/DEP_NAME log worker +[Tue Apr 12 09:15:54 2011] WRK_ID Started Worker (command: 'WORKER_NAME', parameter: '2') +[Tue Apr 12 09:15:54 2011] WRK_ID step: 1 +[Tue Apr 12 09:15:55 2011] WRK_ID step: 2 +[Tue Apr 12 09:15:56 2011] WRK_ID step: 3 +[Tue Apr 12 09:15:57 2011] WRK_ID step: 4 +[Tue Apr 12 09:15:58 2011] WRK_ID step: 5 +[Tue Apr 12 09:15:58 2011] WRK_ID An error occured. Exiting. +[...] +~~~ ## Secure Shell (SSH) @@ -905,7 +1047,5 @@ $ cctrlapp APP_NAME/DEP_NAME deploy --stack [luigi,pinky] [tutorial]: https://www.cloudcontrol.com/blog/best-practice-running-and-analyzing-load-tests-on-your-cloudcontrol-app [Cron Add-on]: https://www.cloudcontrol.com/add-ons/cron [Cron Add-on documentation]: https://www.cloudcontrol.com/dev-center/add-on-documentation/cron -[Worker Add-on]: https://www.cloudcontrol.com/add-ons/worker -[Worker Add-on documentation]: https://www.cloudcontrol.com/dev-center/add-on-documentation/worker [Ubuntu 10.04 LTS Lucid Lynx]: http://releases.ubuntu.com/lucid/ [Ubuntu 12.04 LTS Precise Pangolin]: http://releases.ubuntu.com/precise/