Imagick « Mikko’s blog

Beaucoup de tutoriels au sujet d’ImageMagick (Utile pour le plugin doc2img)

http://valokuva.org/?cat=1

Les articles publiés sur le site

1 | 2 | 3 | 4 | 5 | 6

Working on images asynchronously

15 décembre 2013, par Mikko Koppanen — Imagick, PHP stuff
To get my quota on buzzwords for the day we are going to look at using ZeroMQ and Imagick to create a simple asynchronous image processing system. Why asynchronous? First of all, separating the image handling from a interactive PHP scripts allows us to scale the image processing separately from the web heads. For example we could do the image processing on separate servers, which have SSDs attached and more memory. In this example making the images available to all worker nodes is left to the reader.

Secondly, separating the image processing from a web script can provide more responsive experience to the user. This doesn’t necessarily mean faster, but let’s say in a multiple image upload scenario this method allows the user to do something else on the site while we process the images in the background. This can be beneficial especially in cases where users upload hundreds of images at a time. To achieve a simple distributed image processing infrastructure we are going to use ZeroMQ for communicating between different components and Imagick to work on the images.

The first part we are going to create is a simple “Worker” -process skeleton. Naturally for a live environment you would like to have more error handling and possibly use pcntl for process control, but for the sake of brewity the example is barebones:
php
< view plain text >
1. <?php
3. define ('THUMBNAIL_ADDR', 'tcp://127.0.0.1:5000');
4. define ('COLLECTOR_ADDR', 'tcp://127.0.0.1:5001');
6. class Worker {
8. private $in;
9. private $out;
11. public function __construct ($in_addr, $out_addr)
12. {
13. $context = new ZMQContext ();
15. $this->in = new ZMQSocket ($context, ZMQ::SOCKET_PULL);
16. $this->in->bind ($in_addr);
18. $this->out = new ZMQSocket ($context, ZMQ::SOCKET_PUSH);
19. $this->out->connect ($out_addr);
20. }
22. public function work () {
23. while ($command = $this->in->recvMulti ()) {
24. if (isset ($this->commands [$command [0]])) {
25. echo "Received work" . PHP_EOL;
27. $callback = $this->commands [$command [0]];
29. array_shift ($command);
30. $response = call_user_func_array ($callback, $command);
32. if (is_array ($response))
33. $this->out->sendMulti ($response);
34. else
35. $this->out->send ($response);
36. }
37. else {
38. error_log ("There is no registered worker for {$command [0]}");
39. }
40. }
41. }
43. public function register ($command, $callback)
44. {
45. $this->commands [$command] = $callback;
46. }
47. }
48. ?>
The Worker class allows us to register commands with callbacks associated with them. In our case the Worker class doesn’t actually care or know about the parameters being passed to the actual callback, it just blindly passes them on. We are using two separate sockets in this example, one for incoming work requests and one for passing the results onwards. This allows us to create a simple pipeline by adding more workers in the mix. For example we could first have a watermark worker, which takes the original image and composites a watermark on it, passes the file onwards to thumbnail worker, which then creates different sizes of thumbnails and passes the final results to event collector.

The next part we are going to create a is a simple worker script that does the actual thumbnailing of the images:
php
< view plain text >
1. <?php
2. include __DIR__ . '/common.php';
4. // Create worker class and bind the inbound address to 'THUMBNAIL_ADDR' and connect outbound to 'COLLECTOR_ADDR'
5. $worker = new Worker (THUMBNAIL_ADDR, COLLECTOR_ADDR);
7. // Register our thumbnail callback, nothing special here
8. $worker->register ('thumbnail', function ($filename, $width, $height) {
9. $info = pathinfo ($filename);
11. $out = sprintf ("%s/%s_%dx%d.%s",
12. $info ['dirname'],
13. $info ['filename'],
14. $width,
15. $height,
16. $info ['extension']);
18. $status = 1;
19. $message = '';
21. try {
22. $im = new Imagick ($filename);
23. $im->thumbnailImage ($width, $height);
24. $im->writeImage ($out);
25. }
26. catch (Exception $e) {
27. $status = 0;
28. $message = $e->getMessage ();
29. }
31. return array (
32. 'status' => $status,
33. 'filename' => $filename,
34. 'thumbnail' => $out,
35. 'message' => $message,
36. );
37. });
39. // Run the worker, will block
40. echo "Running thumbnail worker.." . PHP_EOL;
41. $worker->work ();
As you can see from the code the thumbnail worker registers a callback for ‘thumbnail’ command. The callback does the thumbnailing based on input and returns the status, original filename and the thumbnail filename. We have connected our Workers “outbound” socket to event collector, which will receive the results from the thumbnail worker and do something with them. What the “something” is depends on you. For example you could push the response into a websocket to show immediate feeedback to the user or store the results into a database.

Our example event collector will just do a var_dump on every event it receives from the thumbnailer:
php
< view plain text >
1. <?php
2. include __DIR__ . '/common.php';
4. $socket = new ZMQSocket (new ZMQContext (), ZMQ::SOCKET_PULL);
5. $socket->bind (COLLECTOR_ADDR);
7. echo "Waiting for events.." . PHP_EOL;
8. while (($message = $socket->recvMulti ())) {
9. var_dump ($message);
10. }
11. ?>
The final piece of the puzzle is the client that pumps messages into the pipeline. The client connects to the thumbnail worker, passes on filename and desired dimensions:
php
< view plain text >
1. <?php
2. include __DIR__ . '/common.php';
4. $socket = new ZMQSocket (new ZMQContext (), ZMQ::SOCKET_PUSH);
5. $socket->connect (THUMBNAIL_ADDR);
7. $socket->sendMulti (
8. array (
9. 'thumbnail',
10. realpath ('./test.jpg'),
11. 50,
12. 50,
13. )
14. );
15. echo "Sent request" . PHP_EOL;
16. ?>
After this our processing pipeline will look like this:

Now, if we notice that thumbnail workers or the event collectors can’t keep up with the rate of images we are pushing through we can start scaling the pipeline by adding more processes on each layer. ZeroMQ PUSH socket will automatically round-robin between all connected nodes, which makes adding more workers and event collectors simple. After adding more workers our pipeline will look like this:

Using ZeroMQ also allows us to create more flexible architectures by adding forwarding devices in the middle, adding request-reply workers etc. So, the last thing to do is to run our pipeline and see the results:

Let’s create our test image first:
```
$ convert magick:rose test.jpg
```
From the command-line run the thumbnail script:
```
$ php thumbnail.php 
Running thumbnail worker..
```
In a separate terminal window run the event collector:
```
$ php collector.php 
Waiting for events..
```
And finally run the client to send the thumbnail request:
```
$ php client.php 
Sent request
$
```
If everything went according to the plan you should now see the following output in the event collector window:
```
array(4) {
  [0]=>
  string(1) "1"
  [1]=>
  string(56) "/test.jpg"
  [2]=>
  string(62) "/test_50x50.jpg"
  [3]=>
  string(0) ""
}
```
Happy hacking!
Memcached protocol support

15 novembre 2013, par Mikko Koppanen — Imagick
For the past few days I’ve been adding Memcached binary protocol support to PECL memcached extension. The protocol handler provides a high-level abstraction for acting as a memcached server. There are quite a few things still missing and only binary protocol is supported at the moment, but the code seems to work reasonably well in small-scale testing.

I am not sure whether this is useful for anyone, but at least it allows things such as quick prototyping of network servers, exposing sqlite database over memcached protocol etc.

The code is quite simple and implementing a simple server responding to get and set would look roughly like the following:
php
< view plain text >
1. <?php
2. // Create new server instance
3. $server = new MemcachedServer();
5. // Create a simple storage class
6. class Storage {
7. private $values = array ();
9. public function set ($key, $value, $expiration) {
10. $this->values [$key] = array ('value' => $value,
11. 'expires' => time () + $expiration);
12. }
14. public function get ($key) {
15. if (isset ($this->values [$key])) {
16. if ($this->values [$key] ['expires'] < time ()) {
17. unset ($this->values [$key]);
18. return null;
19. }
20. return $this->values [$key] ['value'];
21. }
22. else
23. return null;
24. }
25. }
27. $storage = new Storage ();
29. // Set callback for get command
30. $server->on (Memcached::ON_GET,
31. function ($client_id, $key, &$value, &$flags, &$cas) use ($storage) {
32. echo "Getting key=[$key]" . PHP_EOL;
33. if (($value = $storage->get ($key)) != null)
34. return Memcached::RESPONSE_SUCCESS;
36. return Memcached::RESPONSE_KEY_ENOENT;
37. });
39. // Set callback for set command
40. $server->on (Memcached::ON_SET,
41. function ($client_id, $key, $value, $flags, $expiration, $cas, &$result_cas) use ($storage) {
42. echo "Setting key=[$key] value=[$value]" . PHP_EOL;
43. $storage->set ($key, $value, $expiration);
44. return Memcached::RESPONSE_SUCCESS;
45. });
47. // Run the server on localhost, port 3434. Will block
48. $server->run ("127.0.0.1:3434");
49. ?>
And the client that communicates with the server:
php
< view plain text >
1. <?php
3. $cache = new Memcached();
4. $cache->setOption(Memcached::OPT_BINARY_PROTOCOL, true);
5. $cache->setOption(Memcached::OPT_COMPRESSION, false);
6. $cache->addServer('localhost', 3434);
8. $cache->set ('set_key1', 'This is the first key', 10);
9. var_dump ($cache->get ('set_key1'));
11. $cache->set ('set_key2', 'This is the second key', 2);
12. var_dump ($cache->get ('set_key2'));
13. ?>
The code is still work in progress but it’s available in github: https://github.com/mkoppanen/php-memcached/tree/feature-server. Note that you need to compile libmemcached with –enable-libmemcachedprotocol and the PECL memcached extension with –enable-memcached-protocol.
About image opacity

23 octobre 2013, par Mikko Koppanen — Imagick
There is a common misconception that Imagick::setImageOpacity() would work to reduce the opacity of the image. However, as the name says the method actually sets the opacity throughout the image and thus affects also transparent areas.

To demonstrate let’s first look at this image of a red circle on a transparent background:

Now, let’s apply setImageOpacity on the image:
php
< view plain text >
1. <?php
2. $im = new Imagick ('red-circle.png');
3. $im->setImageOpacity (0.5);
4. $im->writeImage ('red-circle-setopacity.png');
5. ?>
As we can see from the resulting image the transparent background is affected as well.

In order to actually reduce the opacity of the opaque parts Imagick::evaluateImage can be used instead:
php
< view plain text >
1. <?php
2. $im = new Imagick ('red-circle.png');
4. /* Divide the alpha channel value by 2 */
5. $im->evaluateImage(Imagick::EVALUATE_DIVIDE, 2, Imagick::CHANNEL_ALPHA);
6. $im->writeImage ('red-circle-divide.png');
7. ?>
And here are the results:

As the background is already fully transparent so the divide operation causes no changes to it.

Similar example is available in the PHP manual http://php.net/imagick.evaluateimage and I added a note to setImageOpacity page as well (at the time of writing it has not synced to documentation mirrors yet).
PHP extension writing

19 octobre 2013, par Mikko Koppanen — Imagick

I’ve written quite a few PHP extensions over the past years and thought that I could share some of the experience with larger community. PHP internals can be a bit scary at times and in the past I’ve scoured through a lot of extensions to find practical examples of things such as how to return objects from internal functions/methods, how to handle different types of parameters, class properties etc.

To document some of the experiences I started a project called extsample, in which I plan to add practical examples related to extension writing. There won’t be extensive written documentation outside the code, but hopefully the code itself contains enough nuggets of information to be useful. As the README says, if you need a specific example or clarification on something just open an issue in Github.

The project is still very fresh but hopefully soon it will contain more examples. Pull requests are also welcome if you have code that you want to share with others.
More Imagick refresh

4 octobre 2013, par Mikko Koppanen — Imagick, PHP stuff

I’ve committed quite a few changes lately, mainly removing excessive macro usage and making the code more robust. Large amounts of the code was written about six years ago and a lot of things have changed since. Among other things, I’ve probably become a lot better in C.

Under the hood ImagickPixelIterator went through almost a full rewrite, a lot of the internal routines have been renamed and improved and I am happy to say that most of the (useless) macros have been removed.

Some of the user visible/interesting features added recently:

Countable

Imagick class now supports Countable interface and calling count on the object returns amount of images currently in memory. For example for PDF files this is usually the amount of pages. This is purely syntactic sugar as the functionality was available before using the getNumberImages method. The usage of the countable is pretty simple: 021-countable.phpt.

writeImageFile

After tracking down (what I thought was) a bug related to writeImageFile not honouring the format set with setImageFormat I was advised that the format actually depends on the filename. The filename is set during reading the image and just calling setImageFormat and writeImageFile would cause the original file format to be written in the handle.

There is now an additional parameter for writeImageFile for setting the format during the operation. The following test demonstrates the functionality and the issue: 022-writeimagefileformat.phpt.

Memory Management

One of the things that pops up now and then (especially from shared hosting providers) is whether Imagick supports PHP memory limits. Before today the answer was no and you needed to configure ImageMagick separately with reasonable limits.

In the latest master version there is a new compile time flag –enable-imagick-zend-mm, which adds Zend Memory Manager support. This means that Imagick will honour the PHP memory limits and will cause “Out of memory” error to be returned in case of overflow. The following test demonstrates the “usage”: 023-php-allocators.phpt.