Create daemons in PHP Create, daemons, PHP

scyzxp

Everyone knows PHP can be used to create websites. But it can also be used to create desktop applications and commandline tools. And now with a class called System_Daemon, you can even create daemons using nothing but PHP. And did I mention it was easy?
What is a Daemon?

A daemon is a Linux program that run in the background, just like a 'Service' on Windows. It can perform all sorts of tasks that do not require direct user input. Apache is a daemon, so is MySQL. All you ever hear from them is found in somewhere in /var/log, yet they silently power over 40% of the Internet.

You reading this page, would not have been possible without them. So clearly: a daemon is a powerful thing, and can be bend to do a lot of different tasks.
Why PHP?

Most daemons are written in C. It's fast & robust. But if you are in a LAMP oriented company like me, and you need to create a lot of software in PHP anyway, it makes sense:

    * Reuse & connect existing code
      Think of database connections, classes that create customers from your CRM, etc.
    * Deliver new applications very fast
      PHP has a lot of build in functions that speed up development greatly.
    * Everyone knows PHP (right?)
      If you work in a small company: chances are there are more PHP programmers than there are C programmers. What if your C guy abandons ship?

Possible use cases

    * Website optimization
      If you're running a (large) website, jobs that do heavy lifting should be taken away from the user interface and scheduled to run on the machine separately.
    * Log parser
      Continually scan logfiles and import critical messages in your database.
    * SMS daemon
      Read a database queue, and let your little daemon interface with your SMS provider. If it fails, it can easily try again later!
    * Video converter (think Youtube)
      Scan a queue/directory for incoming video uploads. Make some system calls to ffmpeg to finally store them as Flash video files. Surely you don't want to convert video files right after the upload, blocking the user interface that long? No: the daemon will send the uploader a mail when the conversion is done, and proceed with the next scheduled upload

Deamons vs Cronjobs

Some people use cronjobs for the same Possible use cases. Crontab is fine but it only allows you to run a PHP file every minute or so.

    * What if the previous run hasn't finished yet? Overlap can seriously damage your data & cause siginificant load on your machines.
    * What if you can't afford to wait a minute for the cronjob to run? Maybe you need to trigger a process the moment a record is inserted?
    * What if you want to keep track of multiple 'runs' and store data in memory.
    * What if you need to keep your application listening (on a socket for example)

Cronjobs are a bit rude for this, they may spin out of control and don't provide a framework for logging, etc. Creating a daemon would offer more elegance & possibilities. Let's just say: there are very good reasons why a Linux OS isn't composed entirely of Cronjobs
How it works internally

(Nerd alert!) When a daemon program is started, it fires up a second child process, detaches it, and then the parent process dies. This is called forking. Because the parent process dies, it will give the console back and it will look like nothing has happened. But wait: the child process is still running. Even if you close your terminal, the child continues to run in memory, until it either stops, crashes or is killed.

In PHP: forking can be achieved by using the Process Control Extensions. Getting a good grip on it, may take some studying though.
System_Daemon

Because the Process Control Extensions' documentation is a bit rough, I decided to figure it out once, and then wrap my knowledge and the required code inside a PEAR class called: System_Daemon. And so now you can just:

1. require_once "System/Daemon.php";                 // Include the Class
   
2. 
   
3. System_Daemon::setOption("appName", "mydaemon");  // Minimum configuration
   
4. System_Daemon::start();                           // Spawn Deamon!

复制代码

And that's all there is to it. The code after that, will run in your server's background. So next, if you create a while(true) loop around that code, the code will run indefinitely. Remember to build in a sleep(5) to ease up on system resources.
Features & Characteristics

Here's a grab of System_Daemon's features:

    * Daemonize any PHP-CLI script
    * Simple syntax
    * Driver based Operating System detection
    * Unix only
    * Additional features for Debian / Ubuntu based systems like:
    * Automatically generate startup files (init.d)
    * Support for PEAR's Log package
    * Can run with PEAR (more elegance & functionality) or without PEAR (for standalone packages)
    * Default signal handlers, but optionally reroute signals to your own handlers.
    * Set options like max RAM usage

Download

You could download the package from PEAR, or, if you have PEAR installed on your system: just run this from a console:

1. pear install -f System_Daemon

复制代码

You can also update it using that last command.
Alpha

Though I have quite some daemons set up this way, it's officially still 0.5.0-alpha. So please report any bugs over at the PEAR page. Other comments may be posted here.
Complex Example

Ready to dig a little deeper? This example program is called 'logparser', it takes a look at a more complex use of System_Daemon. For instance, it introduces command line arguments like:

1. --no-daemon              # just run the program in the console this time
   
2. --write-initd            # writes a startup file

复制代码

Read this source to get the picture. Don't forget the comments!

1. #!/usr/bin/php -q
   
2. <?php
   
3. /**
   
4. * System_Daemon turns PHP-CLI scripts into daemons.
   
5. *
   
6. * PHP version 5
   
7. *
   
8. * @category  System
   
9. * @package   System_Daemon
   
10. * @author    Kevin <kevin@vanzonneveld.net>
    
11. * @copyright 2008 Kevin van Zonneveld
    
12. * @license   http://www.opensource.org/licenses/bsd-license.php
    
13. * @version   SVN: Release: $Id: logparser.php 215 2009-04-25 10:10:18Z kevin $
    
14. * @link      http://trac.plutonia.nl/projects/system_daemon
    
15. */
    
16. 
    
17. /**
    
18. * System_Daemon Example Code
    
19. *
    
20. * If you run this code successfully, a daemon will be spawned
    
21. * but unless have already generated the init.d script, you have
    
22. * no real way of killing it yet.
    
23. *
    
24. * In this case wait 3 runs, which is the maximum for this example.
    
25. *
    
26. *
    
27. * In panic situations, you can always kill you daemon by typing
    
28. *
    
29. * killall -9 logparser.php
    
30. * OR:
    
31. * killall -9 php
    
32. *
    
33. */
    
34. 
    
35. // Allowed arguments & their defaults
    
36. $runmode = array(
    
37.     "no-daemon" => false,
    
38.     "help" => false,
    
39.     "write-initd" => false
    
40. );
    
41. 
    
42. // Scan command line attributes for allowed arguments
    
43. foreach ($argv as $k=>$arg) {
    
44.     if (substr($arg, 0, 2) == "--" && isset($runmode[substr($arg, 2)])) {
    
45.         $runmode[substr($arg, 2)] = true;
    
46.     }
    
47. }
    
48. 
    
49. // Help mode. Shows allowed argumentents and quit directly
    
50. if ($runmode["help"] == true) {
    
51.     echo "Usage: ".$argv[0]." [runmode]\n";
    
52.     echo "Available runmodes:\n";
    
53.     foreach ($runmode as $runmod=>$val) {
    
54.         echo " --".$runmod."\n";
    
55.     }
    
56.     die();
    
57. }
    
58. 
    
59. // Make it possible to test in source directory
    
60. // This is for PEAR developers only
    
61. ini_set('include_path', ini_get('include_path').':..');
    
62. 
    
63. // Include Class
    
64. error_reporting(E_ALL);
    
65. require_once "System/Daemon.php";
    
66. 
    
67. // Setup
    
68. $options = array(
    
69.     "appName" => "logparser",
    
70.     "appDir" => dirname(__FILE__),
    
71.     "appDescription" => "Parses vsftpd logfiles and stores them in MySQL",
    
72.     "authorName" => "Kevin van Zonneveld",
    
73.     "authorEmail" => "kevin@vanzonneveld.net",
    
74.     "sysMaxExecutionTime" => "0",
    
75.     "sysMaxInputTime" => "0",
    
76.     "sysMemoryLimit" => "1024M",
    
77.     "appRunAsGID" => 1000,
    
78.     "appRunAsUID" => 1000
    
79. );
    
80. 
    
81. System_Daemon::setOptions($options);
    
82. 
    
83. // Overrule the signal handler with any function
    
84. System_Daemon::setSigHandler(SIGCONT, array("System_Daemon",
    
85.     "defaultSigHandler"));
    
86. 
    
87. 
    
88. // This program can also be run in the forground with runmode --no-daemon
    
89. if (!$runmode["no-daemon"]) {
    
90.     // Spawn Daemon
    
91.     System_Daemon::start();
    
92. }
    
93. 
    
94. // With the runmode --write-initd, this program can automatically write a
    
95. // system startup file called: 'init.d'
    
96. // This will make sure your daemon will be started on reboot
    
97. if (!$runmode["write-initd"]) {
    
98.     System_Daemon::log(System_Daemon::LOG_INFO, "not writing ".
    
99.         "an init.d script this time");
    
100. } else {
     
101.     if (($initd_location = System_Daemon::writeAutoRun()) === false) {
     
102.         System_Daemon::log(System_Daemon::LOG_NOTICE, "unable to write ".
     
103.             "init.d script");
     
104.     } else {
     
105.         System_Daemon::log(System_Daemon::LOG_INFO, "sucessfully written ".
     
106.             "startup script: ".$initd_location);
     
107.     }
     
108. }
     
109. 
     
110. // Run your code
     
111. // Here comes your own actual code
     
112. 
     
113. // This variable gives your own code the ability to breakdown the daemon:
     
114. $runningOkay = true;
     
115. 
     
116. // This variable keeps track of how many 'runs' or 'loops' your daemon has
     
117. // done so far. For example purposes, we're quitting on 3.
     
118. $cnt = 1;
     
119. 
     
120. // While checks on 3 things in this case:
     
121. // - That the Daemon Class hasn't reported it's dying
     
122. // - That your own code has been running Okay
     
123. // - That we're not executing more than 3 runs
     
124. while (!System_Daemon::isDying() && $runningOkay && $cnt <=3) {
     
125.     // What mode are we in?
     
126.     $mode = "'".(System_Daemon::isInBackground() ? "" : "non-" ).
     
127.         "daemon' mode";
     
128.    
     
129.     // Log something using the Daemon class's logging facility
     
130.     // Depending on runmode it will either end up:
     
131.     //  - In the /var/log/logparser.log
     
132.     //  - On screen (in case we're not a daemon yet)  
     
133.     System_Daemon::log(System_Daemon::LOG_INFO,
     
134.         System_Daemon::getOption("appName").
     
135.         " running in ".$mode." ".$cnt."/3");
     
136.    
     
137.     // In the actuall logparser program, You could replace 'true'
     
138.     // With e.g. a  parseLog('vsftpd') function, and have it return
     
139.     // either true on success, or false on failure.
     
140.     $runningOkay = true;
     
141.     //$runningOkay = parseLog('vsftpd');
     
142.    
     
143.     // Should your parseLog('vsftpd') return false, then
     
144.     // the daemon is automatically shut down.
     
145.     // An extra log entry would be nice, we're using level 3,
     
146.     // which is critical.
     
147.     // Level 4 would be fatal and shuts down the daemon immediately,
     
148.     // which in this case is handled by the while condition.
     
149.     if (!$runningOkay) {
     
150.         System_Daemon::log(System_Daemon::LOG_ERR, "parseLog() ".
     
151.             "produced an error, ".
     
152.             "so this will be my last run");
     
153.     }
     
154.    
     
155.     // Relax the system by sleeping for a little bit
     
156.     // iterate also clears statcache
     
157.     System_Daemon::iterate(2);
     
158. 
     
159.     $cnt++;
     
160. }
     
161. 
     
162. // Shut down the daemon nicely
     
163. // This is ignored if the class is actually running in the foreground
     
164. System_Daemon::stop();
     
165. ?>

复制代码

Console action: Controlling the Daemon

Now that we've created an example daemon, it's time to fire it up! I'm going to assume the name of your daemon is logparser. This can be changed with the statement: System_Daemon::setOption("appName", "logparser"). But the name of the daemon is very important because it is also used in filenames (like the logfile).
Execute

Just make your daemon script executable, and then execute it:

1. chmod a+x ./logparser.php
   
2. ./logparser.php

复制代码

Check

Your daemon has no way of communicating through your console, so check for messages in:

1. tail /var/log/logparser.log

复制代码

And see if it's still running:

1. ps uf -C logparser.php

复制代码

Kill

Without the start/stop files (see below for howto), you need to:

1. killall -9 logparser.php

复制代码

Autch.. Let's work on those start / stop files, right?
Start / Stop files (Debian & Ubuntu only)

Real daemons have an init.d file. Remember you can restart Apache with the following statement?

1. /etc/init.d/apache2 restart

复制代码

That's a lot better than killing. So you should be able to control your own daemon like this as well:

1. /etc/init.d/logparser stop
   
2. /etc/init.d/logparser start

复制代码

Well with System_Daemon you can write autostartup files using the writeAutoRun() method, look:

1. $path = System_Daemon::writeAutoRun();

复制代码

On success, this will return the path to the autostartup file: /etc/init.d/logparser, and you're good to go!
Run on boot

Surely you want your daemon to run at system boot..  So on Debian & Ubuntu you could type:

1. update-rc.d logparser defaults

复制代码

Done your daemon now starts every time your server boots. Cancel it with:

1. update-rc.d -f logparser remove

复制代码

Troubleshooting

Here are some issues you may encounter down the road.

    * Don't use echo()
      Echo writes to the STDOUT of your current session. If you logout, that will cause fatal errors and the daemon to die. Use System_Daemon::log() instead.
    * Connect to MySQL after you start() the daemon.
      Otherwise only the parent process will have a MySQL connection, and since that dies.. It's lost and you will get a 'MySQL has gone away' error.
    * Error handling
      Good error handling is imperative. Daemons are often mission critical applications and you don't want an uncatched error to bring it to it's knees.
          o Reconnect to MySQL
            A connection may be interrupted. Think about network downtime or lock-ups when your database server makes backups. Whatever the cause: You don't want your daemon to die for this, let it try again later.
          o PHP error handler
            As of 0.6.3, System_Daemon forwards all PHP errors to the log() method, so keep an eye on your logfile. This behavior can be controlled using the logPhpErrors (true||false) option.
    * Monit
      Monit is a standalone program that can kickstart any daemon, based on your parameters. Should your daemon fail, monit will mail you and try to restart it.
    * Watch that memory
      Some classes keep a history of executed commands, sent mails, queries, whatever. They were designed without knowing they would ever be used in a daemonized environment.
      Cause daemons run indefinitely this 'history' will expand indefinitely. Since unfortunately your server's RAM is not infinite, you will run into problems at some point.
      This makes it's very important to address these memory 'leaks' when building daemons.
    * Statcache will corrupt your data
      If you do a file_exists(), PHP remembers the results to ease on your disk until the process end. That's ok but since the Daemon process does not end, PHP will not be able to give you up to date information. As of 0.8.0 you should call System_Daemon::iterate(2) instead of e.g. sleep(2), this will sleep & clear the cache and give you fresh & valid data.

I know I'm saying MySQL a lot, but you can obviously replace that with Oracle, MSSQL, PostgreSQL, etc.



http://kevin.vanzonneveld.net/te ... ate_daemons_in_php/

scyzxp

http://pear.php.net/package/System_Daemon/