Uploaded image for project: 'CMS'
  1. CMS
  2. CMS-4404

Cache statistics monitoring tool.

      Here is a patch containing a new feature which is a cache statistics monitoring tool.
      Currently it adds tow table in the admin workspace. One is focused on the back-office statistics (pages and page elements). The other is focused on the efficiency of the caches of Front servers (Apache and Ametys Site).

      To install it, we need to create some SQL table in the database of your choice (plugin-web/scripts/mysql/jdbc_cache_mon.sql), and then configure the new parameters in the your Ametys applications (Front and Back).

      The import of the raw data (request on resources/pages) and the update of the statistics is made every hours.

      There are two patches, one for the WEB, the other for the SITE application. There is also a zip archive containing the source code of the modified file for the WEB (equivalent to the patch) plus the added binary resources (images).

        1. cache-monitoring-SITE_20130408_v0.1.patch
          86 kB
          Thibaut Rizzi
        2. cache-monitoring-SITE_20130426_v0.2.patch
          62 kB
          Thibaut Rizzi
        3. cache-monitoring-SITE_20130523_v0.3.patch
          13 kB
          Thibaut Rizzi
        4. cache-monitoring-WEB_20130408_v0.1.patch
          257 kB
          Thibaut Rizzi
        5. cache-monitoring-WEB_20130426_v0.2.patch
          237 kB
          Thibaut Rizzi
        6. cache-monitoring-WEB_20130523_v0.3.2.patch
          6 kB
          Thibaut Rizzi
        7. cache-monitoring-WEB_20130523_v0.3.3.patch
          46 kB
          Thibaut Rizzi
        8. cache-monitoring-WEB_20130523_v0.3.patch
          80 kB
          Thibaut Rizzi

          [CMS-4404] Cache statistics monitoring tool.

          InputDataGenerator and PageGenerator can sometimes lead to an NPE.
          I add another patch.

          Thibaut Rizzi (Inactive) added a comment - InputDataGenerator and PageGenerator can sometimes lead to an NPE. I add another patch.

          TODO :

          • document install
          • document exploit

          Raphaël Franchet added a comment - TODO : document install document exploit

          • purge old unprocessed lines (more than 24 hours old)

          Done. Cf the new patch.

          Thibaut Rizzi (Inactive) added a comment - purge old unprocessed lines (more than 24 hours old) Done. Cf the new patch.

          TODO :

          • document install
          • document exploit
          • purge old unprocessed lines (more than 24 hours old)

          Raphaël Franchet added a comment - TODO : document install document exploit purge old unprocessed lines (more than 24 hours old)

          TODO :

          • add new tables to the demo repository
          • document install
          • document exploit
          • purge old unprocessed lines (more than 24 hours old)

          Raphaël Franchet added a comment - TODO : add new tables to the demo repository document install document exploit purge old unprocessed lines (more than 24 hours old)

          TODO :

          • finish new table structure and code to handle long paths

          Here is a patch for this

          Thibaut Rizzi (Inactive) added a comment - TODO : finish new table structure and code to handle long paths Here is a patch for this

          Raphaël Franchet added a comment - - edited

          Installation

          new sql tables

          and new apache conf

          LoadModule unique_id_module modules/mod_unique_id.so
          LoadModule headers_module modules/mod_headers.so

          RewriteRule ^(/.*)$ http://localhost:PORT_TOMCAT$1 [P,L]
          ProxyPassReverse / http://localhost:PORT_TOMCAT/

          becomes

          RewriteRule ^(/.*)$ http://localhots:PORT_TOMCAT$1 [E=AMETYS_PROXY_PASS:1,P,L]
          RequestHeader set UNIQUE_ID %{UNIQUE_ID}e env=AMETYS_PROXY_PASS
          ProxyPassReverse / http://localhost:PORT_TOMCAT/

          and

          LogFormat "%{UNIQUE_ID}e INTERNAL_SITE_NAME %h %l %u %t \"%r\" %s/%>s %b %{AMETYS_PROXY_PASS}e \"%{Referer}i\" \"%{User-Agent}i\"" INTERNALSITENAME-combined

          CustomLog /path/to/log/file INTERNALSITENAME-combined

          Raphaël Franchet added a comment - - edited Installation new sql tables and new apache conf LoadModule unique_id_module modules/mod_unique_id.so LoadModule headers_module modules/mod_headers.so RewriteRule ^(/.*)$ http://localhost:PORT_TOMCAT$1 [P,L] ProxyPassReverse / http://localhost:PORT_TOMCAT/ becomes RewriteRule ^(/.*)$ http://localhots:PORT_TOMCAT$1 [E=AMETYS_PROXY_PASS:1,P,L] RequestHeader set UNIQUE_ID %{UNIQUE_ID}e env=AMETYS_PROXY_PASS ProxyPassReverse / http://localhost:PORT_TOMCAT/ and LogFormat "%{UNIQUE_ID}e INTERNAL_SITE_NAME %h %l %u %t \"%r\" %s/%>s %b %{AMETYS_PROXY_PASS}e \"%{Referer}i\" \"%{User-Agent}i\"" INTERNALSITENAME-combined CustomLog /path/to/log/file INTERNALSITENAME-combined

          Raphaël Franchet added a comment - - edited

          Patch has been integrated to the SITE project (and default parameters where added to the conf)

          TODO :

          • finish new table structure and code to handle long paths
          • add new tables to the demo repository
          • document install
          • document exploit

          Raphaël Franchet added a comment - - edited Patch has been integrated to the SITE project (and default parameters where added to the conf) TODO : finish new table structure and code to handle long paths add new tables to the demo repository document install document exploit

          Raphaël Franchet added a comment - - edited

          Patch has been integrated to the WEB project (and default parameters where added to the conf)

          TODO :

          • integrate patch on the front
          • finish new table structure and code to handle long paths
          • add new tables to the demo repository

          Raphaël Franchet added a comment - - edited Patch has been integrated to the WEB project (and default parameters where added to the conf) TODO : integrate patch on the front finish new table structure and code to handle long paths add new tables to the demo repository

          Problem with the page_path that must handle very long path:

          • Cache_RA_HTTPServer.Path
          • Cache_RA_HTTPServer.Query_String
          • Cache_RA_Front.Ametys_Path
          • Cache_RA_Back.Page_Path
          • Cache_Stats_Front.Server_Path
          • Cache_Stats_Front.Front_Path
          • Cache_Stats_Back.Page_Path

          check also, ids : there is no need to limit them

          Raphaël Franchet added a comment - Problem with the page_path that must handle very long path: Cache_RA_HTTPServer.Path Cache_RA_HTTPServer.Query_String Cache_RA_Front.Ametys_Path Cache_RA_Back.Page_Path Cache_Stats_Front.Server_Path Cache_Stats_Front.Front_Path Cache_Stats_Back.Page_Path check also, ids : there is no need to limit them

            raphael Raphaël Franchet
            trizzi Thibaut Rizzi (Inactive)
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

              Created:
              Updated:
              Resolved: