CacheAccelerator for MODx Evo. Reducing significantly the number of queries to the database by caching dynamic snippets
Hello. I recently met with MODx CMF. Master currently version Evolution. The whole system is quite nice and very flexible, however, after reading closer, I found a number of shortcomings. Some of them did not give me any rest and leave it as is, I could not.
Stop at one of the most sensitive criteria of any CMS/CMF — performance.
In General, performance in MODx all the rules. He is written well-enough optimized. Moreover, due to its flexibility gives the developer the ability to manage bottlenecks in the projects being undertaken.
However, I was just shocked by the processing method of the display of news with the help of Ditto, comments with Jot, and so on. Namely, the need to disable caching for the whole page have a Ditto (because of problems with PHx), and call the snippet from Jot.
After all, if records quite a lot, on one page they will not fit, and this means that, for example, the news feed should be split into multiple pages. But if MODx caching is enabled on this page, at the transition between the parts of the news, we see all the same content that the first hit the cache!
Suggest what official sources?
They suggest that the snippets work with multiple
pages never cached.
Page call to the snippet is not cached "page setup => Cached => Off"
Page call to the snippet is cached "page setup => Cached => On"
As we can see, caching means MODx in both cases is not possible. No caching directly affects the speed of the system, because each time you call the page, all the data of a snippet collected from the database.
In my tests, the page call, Ditto with 10 records per page displaying a title, a brief abstract and date of publication, generates about 11 queries to the database. The use of additional filters, search conditions in fields of TV, will give even more depressing statistics.
demo
The same thing happens on the calling page Jot.
Call Jot with 10 comments per page, standard information about each
comments and form for adding new messages, and creates order
22 — 24 queries to the database.
demo
It's not critical as long as the base is very small, but with the growth of the base is the number of queries will increase the response time exponentially relative to the number of documents.
After some thought, I wrote CacheAccelerator consisting of one module, one snippet and third-party library.
Cache Accelerator, by caching the issuance of any arbitrary snippet (apart from Ditto and Jot), increases system speed and reduces the number of queries to the database.
In our tests, a second request to the page discussed above with Ditto, gives the result of reducing the number of requests to the database from 11 to 3(!), page with Jot, from 22 to 1(!).
This product does not use any hack methods of integration into the MODx engine. Is a completely separate code block. Very easy to install and use.
First fileCache downloaded from the page:
http://neo22s.com/filecache/
or direct link:
http://lab.neo22s.com/fileCache/fileCache.zip
Create a directory /assets/plugins/cacheaccelerator
In the directory cacheaccelerator create the directory cache (/assets/plugins/cacheaccelerator/cache)
Next, from the downloaded archive copy the file fileCache.php in the directory /assets/plugins/cacheaccelerator
Then, in the MODx Manager, click -> Manage -> Snippets -> New snippet. Create a new snippet with the name CacheAccelerator.
There copy the contents of the snippet.
Store the generated snippet. Then go to the tab Plugins.
Click Create a plugin.
Ask him the name CacheAcceleratorClear. Copy the contents of the plugin.
Attention! After I copied the content of the plugin, go to the tab System events, in which check boxes the event OnCacheUpdate in the Cache Service Events!
Then click Save.
All installation CacheAccelerator completed.
This plugin does one thing. When MODx clears its cache, it also clears the cache CacheAccelerator
CacheAccelerator and can be applied to any of the snippets. To cache any results.
I will consider its application on examples of calls of Ditto and Jot.
Call Ditto will look like the following:
Instead of the call itself Ditto, the snippet is invoked CacheAccelerator, and the name of the cached snippet (in this case Ditto) specified in the parameter snippetToCache.
This is followed by the parameter cacheId. It is designed to separate the cached content in case of presence of several cacheable snippets on the page. For example, the left menu, latest news, and the news feed. Can contain any value, which will bind the cached content.
Note that if Ditto was called in double brackets [[]], then this call must also be contained in them.
Example call Jot:
Himself cached snippet is also specified in the parameter snippetToCache and cacheId the identifier of the cached content on the page.
Here there are two parameters dropCacheField and noCacheGroups. We discuss in more detail.
The fact that, unlike the news, adding which comes from the Manager of MODx and where, after adding each of the news, cleared the cache in the comments Jot any user who visits the site can add a comment. At the same time that this comment was visible both to him and to others, you must clear the cache CacheAccelerator. This action may be necessary not only for Jot, but for many other snippets, the results of which I would like to cache, but also provide the ability to update the cache after any user action.
For these purposes, and serves as a parameter dropCacheField.
It contains a list of conditions separated by ||. With the operation of any of these conditions is clear the cache CacheAccelerator.
the Condition can be the field name.
Upon detection of which in GET requests or POST cache will be updated.
Can be compared.
Using semicolons in lists:
If the name field is blank, the comparison is applied to all existing fields.
Here is an example:
It says here:
the
Any coincidence, have cleaned the cache.
When posting a new message in Jot, is transmitted in a form contains a field called JotForm. Thus, after posting the comment, the cache will be cleared and the information will remain relevant. After the first request to this page, the cache is again created and the subsequent treatment will be given from the cache.
NoCacheGroups parameter contains a comma-delimited list | | which contains the group of web users, the return data from the cache which will not be made, and the snippet will execute on every call. Because the moderators, for example, the shape of the output is different with the buttons and when getting such a request to the cache other advanced users will also see a form with elements that are not relevant to their query.
Clear cache CacheAccelerator also from anywhere by calling:
chunk:
snippet:
Any placeholders that are set cached snippet will also be cached and written when called from the cache.
Example:
List of parameters:
snippetToCache — the name of the snippet to cache, such as `Ditto`.
cacheId — the ID of a cached code snippet on the page, for example `News`
dropCacheField — a list of fields and conditions for them under which resets the cache, such as `JotForm||post;true;2`
noCacheGroups — the list of groups to which the processing caching will not be made, such as `admins||moderators`
clearCache — if set to 1, is forced cache clearing.
logMessages — if set to 1, before the contents of the snippet will display the system message about the hit in the cache and so on.
List of terms:
1 != Not equal
2 = Is
3 < Less than
4 > More than
5 <= Less than or equal to
6 >= Greater than or equal
7 Contains
8 does Not contain
Download ready version CacheAccelerator You can the website.
I in any case do not claim the completeness of the product and the absence of errors in his work. This is the first alpha version, besides, I have little experience in writing such guidelines.
Any help would be appreciated in the development and elimination of faults and errors.
Added new parameters to the snippet:
noCacheRoles — list of roles of managers for which handling of caching will not be performed
for example `Administrator||Editor
checkURL is to create a separate cache for different URLS (1/0). Enabled by default. Include useful for snippets with page navigation.
As well as a new parameter for plugin
only_manual resolution only manual reset of the cache.
For revision thank you Andchir!
Download the new version, you can still here or otsuda
Article based on information from habrahabr.ru
Stop at one of the most sensitive criteria of any CMS/CMF — performance.
In General, performance in MODx all the rules. He is written well-enough optimized. Moreover, due to its flexibility gives the developer the ability to manage bottlenecks in the projects being undertaken.
However, I was just shocked by the processing method of the display of news with the help of Ditto, comments with Jot, and so on. Namely, the need to disable caching for the whole page have a Ditto (because of problems with PHx), and call the snippet from Jot.
After all, if records quite a lot, on one page they will not fit, and this means that, for example, the news feed should be split into multiple pages. But if MODx caching is enabled on this page, at the transition between the parts of the news, we see all the same content that the first hit the cache!
Suggest what official sources?
They suggest that the snippets work with multiple
pages never cached.
examples of a call to Ditto and Jot:
Page call to the snippet is not cached "page setup => Cached => Off"
[[Ditto?
&parents=`1`
&display=`10`
&paginate=`1`
&paginateAlwaysShowLinks=`1`
]]Page call to the snippet is cached "page setup => Cached => On"
[!Jot?
&customfields=`name,email`
&pagination=`10`
&badwords=`*****`
&canmoderate=`Site Admins`
&captcha=`1`!]
!]As we can see, caching means MODx in both cases is not possible. No caching directly affects the speed of the system, because each time you call the page, all the data of a snippet collected from the database.
In my tests, the page call, Ditto with 10 records per page displaying a title, a brief abstract and date of publication, generates about 11 queries to the database. The use of additional filters, search conditions in fields of TV, will give even more depressing statistics.
demo
The same thing happens on the calling page Jot.
Call Jot with 10 comments per page, standard information about each
comments and form for adding new messages, and creates order
22 — 24 queries to the database.
demo
It's not critical as long as the base is very small, but with the growth of the base is the number of queries will increase the response time exponentially relative to the number of documents.
After some thought, I wrote CacheAccelerator consisting of one module, one snippet and third-party library.
Cache Accelerator, by caching the issuance of any arbitrary snippet (apart from Ditto and Jot), increases system speed and reduces the number of queries to the database.
In our tests, a second request to the page discussed above with Ditto, gives the result of reducing the number of requests to the database from 11 to 3(!), page with Jot, from 22 to 1(!).
This product does not use any hack methods of integration into the MODx engine. Is a completely separate code block. Very easy to install and use.
so, setting:
First fileCache downloaded from the page:
http://neo22s.com/filecache/
or direct link:
http://lab.neo22s.com/fileCache/fileCache.zip
Create a directory /assets/plugins/cacheaccelerator
In the directory cacheaccelerator create the directory cache (/assets/plugins/cacheaccelerator/cache)
Next, from the downloaded archive copy the file fileCache.php in the directory /assets/plugins/cacheaccelerator
Then, in the MODx Manager, click -> Manage -> Snippets -> New snippet. Create a new snippet with the name CacheAccelerator.
There copy the contents of the snippet.
Snippet CacheAccelerator:
<?php
//The comparison function. Works in accordance with the comparison function Ditto
if(!function_exists(cacheFieldsCompare)) {
cacheFieldsCompare function ($param1, $param2, $param3){
/*
1 or != Not equal
2 or = Is
3 or < Less than
4 or > More than
5 or <= Less than or equal to
6 or >= Greater than or equal
7 Contains
8 does Not contain
*/
switch($param3){
case 1:
return $param1 != $param2;
break;
case 2:
return $param1 == $param2;
break;
case 3:
return $param1 < $param2;
break;
case 4:
return $param1 > $param2;
break;
case 5:
return $param1 <= $param2;
break;
case 6:
return $param1 >= $param2;
break;
case 7:
return stristr($param1, $param2);
break;
case 8:
return !stristr($param1, $param2);
}
}
}
$nocache = isset($nocache)? $nocache : 0; //flag the need to reset cache
$url = $_SERVER["REQUEST_URI"]; //the current url, is included in the cache key
$path_to_cacheengine=$modx- > config['base_path']."assets/plugins/cacheaccelerator/"; //path to directory with CacheAccelerator
$path_to_cache=$modx- > config['base_path']."assets/plugins/cacheaccelerator/cache/"; //path to directory to store the cache (can be pozvolim)
require_once ($path_to_cacheengine."fileCache.php"); //request class fileCache
$cache = fileCache::GetInstance(84600*7,$path_to_cache);//create instance of class fileCache
//handling of flag forced cache cleanup
if((int)$clearCache){
if($logMessages) echo("Clearing cache...");
$cache- > deleteCache(0);
return;
}
//handling user groups for which caching is not performed (site administrators, moderators, etc.)
$noCacheGroups = isset($noCacheGroups) ? $noCacheGroups : "";
$nocache = intval($modx- > isMemberOfWebGroup(explode("||",$nocacheGroups)) || $modx- > checkSession()) ? 2 : $nocache;
if($nocache == 2){
if($logMessages) echo("No caching for this web group.");
}
/* stop processing fields, giving a signal to reset the cache. in the case of coincidence of the conditions, the cache is reset */
if(isset($dropCacheField)){
$fieldsArray = explode("||", $dropCacheField);
foreach ($fieldsArray as $field){
$field1 = explode(";", $field);
if($field1[1] && $field[2]){
if(empty($field1[0])){
foreach ($_POST as $key => $postField){
if(cacheFieldsCompare($postField, $field1[1], $field1[2])){
$nocache = 1;
continue;
}
}
foreach ($_GET as $key => $getField){
if(cacheFieldsCompare($getField, $field1[1], $field1[2])){
$nocache = 1;
continue;
}
}
} else {
if(!empty($_POST[$field1[0]])){
if(cacheFieldsCompare($_POST[$field1[0]], $field1[1], $field1[2])){
$nocache = 1;
continue;
}
}
if(!empty($_GET[$field1[0]])){
if(cacheFieldsCompare($_GET[$field1[0]], $field1[1], $field1[2])){
$nocache = 1;
continue;
}
}
}
} else {
if(!empty($_POST[$field1[0]]) || !empty($_GET[$field1[0]]))
$nocache = 1;
}
}
//directly, the cache is reset at the coincidence of conditions
if($nocache == 1){
if($logMessages) echo("Clearing cache...");
$cache- > deleteCache(0);
}
}
//request the result of the snippet from the cache
if($nocache == 0){
$cached = $cache->cache($cacheId.$url);
if(isset($cached)){
if($logMessages) echo("Cache hit!");
$modx- > placeholders = $cached['placeholders']; //setting the placeholders of a cached code snippet
return $cached['content']; //return the result of the snippet from the cache
}
}
$output = $modx- > runSnippet($snippetToCache, $modx->event->params); //direct snippet for the settings
//cache the result of the snippet
if($nocache == 0){
if($logMessages) echo("Storing to cache...");
$cache->cache($cacheId.$url,array('placeholders' => $modx- > placeholders, 'content' => $output));
}
//return the result of the snippet in the MODx parser
return($output);
?>Store the generated snippet. Then go to the tab Plugins.
Click Create a plugin.
Ask him the name CacheAcceleratorClear. Copy the contents of the plugin.
the Plugin CacheAcceleratorClear:
$path_to_cacheengine=$modx- > config['base_path']."assets/plugins/cacheaccelerator/"; //path to directory with CacheAccelerator
$path_to_cache=$modx- > config['base_path']."assets/plugins/cacheaccelerator/cache/"; //path to directory to store the cache (can be pozvolim, but must match the value of a snippet CacheAccelerator)
$cache = fileCache::GetInstance(84600*7,$path_to_cache);//create instance of class fileCache
$cache- > deleteCache(0);//clear cache
return;Attention! After I copied the content of the plugin, go to the tab System events, in which check boxes the event OnCacheUpdate in the Cache Service Events!
Then click Save.
All installation CacheAccelerator completed.
This plugin does one thing. When MODx clears its cache, it also clears the cache CacheAccelerator
Using CacheAccelerator:
CacheAccelerator and can be applied to any of the snippets. To cache any results.
I will consider its application on examples of calls of Ditto and Jot.
Call Ditto will look like the following:
[[CacheAccelerator?
&snippetToCache=`Ditto`
&cacheId=`News`
&parents=`1`
&display=`10`
&paginate=`1`
&paginateAlwaysShowLinks=`1`
]]Instead of the call itself Ditto, the snippet is invoked CacheAccelerator, and the name of the cached snippet (in this case Ditto) specified in the parameter snippetToCache.
This is followed by the parameter cacheId. It is designed to separate the cached content in case of presence of several cacheable snippets on the page. For example, the left menu, latest news, and the news feed. Can contain any value, which will bind the cached content.
Note that if Ditto was called in double brackets [[]], then this call must also be contained in them.
Example call Jot:
[!CacheAccelerator?
&snippetToCache=`Jot`
&cacheId=`Comments`
&dropCacheField=`JotForm||post;true;2||;publish;2||;unpublish;2||;delete;2||;edit;2`
&noCacheGroups=`Site Admins`
&customfields=`name,email`
&pagination=`10`
&badwords=`*****`
&canmoderate=`Site Admins`
&captcha=`1`
!]Himself cached snippet is also specified in the parameter snippetToCache and cacheId the identifier of the cached content on the page.
Here there are two parameters dropCacheField and noCacheGroups. We discuss in more detail.
The fact that, unlike the news, adding which comes from the Manager of MODx and where, after adding each of the news, cleared the cache in the comments Jot any user who visits the site can add a comment. At the same time that this comment was visible both to him and to others, you must clear the cache CacheAccelerator. This action may be necessary not only for Jot, but for many other snippets, the results of which I would like to cache, but also provide the ability to update the cache after any user action.
For these purposes, and serves as a parameter dropCacheField.
It contains a list of conditions separated by ||. With the operation of any of these conditions is clear the cache CacheAccelerator.
the Condition can be the field name.
Upon detection of which in GET requests or POST cache will be updated.
Can be compared.
Using semicolons in lists:
field;value;metateIf the name field is blank, the comparison is applied to all existing fields.
Here is an example:
&dropCacheField=`JotForm||post;true;2||;delete;2`It says here:
the
-
the
- If in the query there is a field JotForm the
- If there is a field post and it is true the
- If any field has the value delete
Any coincidence, have cleaned the cache.
When posting a new message in Jot, is transmitted in a form contains a field called JotForm. Thus, after posting the comment, the cache will be cleared and the information will remain relevant. After the first request to this page, the cache is again created and the subsequent treatment will be given from the cache.
NoCacheGroups parameter contains a comma-delimited list | | which contains the group of web users, the return data from the cache which will not be made, and the snippet will execute on every call. Because the moderators, for example, the shape of the output is different with the buttons and when getting such a request to the cache other advanced users will also see a form with elements that are not relevant to their query.
Clear cache CacheAccelerator also from anywhere by calling:
chunk:
[!CacheAccelerator? &clearCache=`1`!]snippet:
$modx- > runSnippet("CacheAccelerator", array("clearCache" = > 1))Any placeholders that are set cached snippet will also be cached and written when called from the cache.
Example:
[[CacheAccelerator?&snippetToCache=`Ditto`&cacheId=`News`]]
It is shown [+start+] - [+stop+] of [+total+] News<br>
[+previous+] [+pages+] [+next+]<br>List of parameters:
snippetToCache — the name of the snippet to cache, such as `Ditto`.
cacheId — the ID of a cached code snippet on the page, for example `News`
dropCacheField — a list of fields and conditions for them under which resets the cache, such as `JotForm||post;true;2`
noCacheGroups — the list of groups to which the processing caching will not be made, such as `admins||moderators`
clearCache — if set to 1, is forced cache clearing.
logMessages — if set to 1, before the contents of the snippet will display the system message about the hit in the cache and so on.
List of terms:
1 != Not equal
2 = Is
3 < Less than
4 > More than
5 <= Less than or equal to
6 >= Greater than or equal
7 Contains
8 does Not contain
Download ready version CacheAccelerator You can the website.
I in any case do not claim the completeness of the product and the absence of errors in his work. This is the first alpha version, besides, I have little experience in writing such guidelines.
Any help would be appreciated in the development and elimination of faults and errors.
UPD.
Added new parameters to the snippet:
noCacheRoles — list of roles of managers for which handling of caching will not be performed
for example `Administrator||Editor
checkURL is to create a separate cache for different URLS (1/0). Enabled by default. Include useful for snippets with page navigation.
As well as a new parameter for plugin
only_manual resolution only manual reset of the cache.
For revision thank you Andchir!
Download the new version, you can still here or otsuda
