v2.11 updates

Amended README with context and background while cleaning up historical items.

README.md

@@ -1,25 +1,70 @@
+## Introduction ##
+Over the years I've been using Zabbix as both personal and business solution for monitoring. One of the missing features in Zabbix itself was the ability to have graphs generated and associated to monitoring messages as well as an easy way to jump back into Zabbix when searching for monitoring information for specific events, problems or to have a direct link to the associated Zabbix item or host.
+
+This initiated the development of mailGraph v1 in Zabbix 5.4 delivering an elementary solution for sending HTML enriched messages from Zabbix including graphs.
+To facilitate in a very flexible way to setup and format messages including the graphs, Twig made its introduction towards mailGraph v2, the current release branch of mailGraph.
+
+The below message is just an example of what MailGraph is capable of. The template engine in Twig allows for a fully customized message creation to your needs! It is also possible to add more Zabbix fields as any field is passed to Twig when accessible in Zabbix through the macro mechanism. Example: "{ITEM.ID}".
+
+Example message:
+[![](images/Example-mail-message-v122.png?raw=true)](images/Example-mail-message-v122.png)
+
+mailGraph is capable of adding several series of graphs into a single message delivering a unique experience when and how groups of graph images per the requested periods of time are added.
+Currently mailGraph supports hosts, (one or more related) items and screens (applicable to Zabbix 5.4 only).
+
+More information can be found in the Wiki.
+
+## Installation ##
+Please refer to the Wiki how to get mailGraph installed and configured on your system.
+
+## mailGraph v2.11 release ##
+_(2023/07/01)_
+
+_This version has been verified with Zabbix 5.4 and 6.0 LTS and is expected to work with 6.4 and later (based on v2.10 testing)_
+
+Release notes
+- Added pre- and postchecking of variables to the Zabbix javascript - this will prevent the 'invalid JSON' messages and provide better feedback for errors
+- When testing MailGraph it is now possible to set the eventId to zero - a random problem will be picked up via the API
+- Zabbix Media Type XML reverted back to version 5.4 (for backwards compatibility)
+
+Modified files
+- mailGraph.php
+- mailGraph.xml
+- javascript/zabbix.mailGraph.js
+
+For those upgrading to the latest release without installing the media type:
+- copy new mailGraph.php over existing mailGraph.php
+- open the Media type MailGraph in Zabbix and edit the javascript
+-- replace the script contents with the contents of javascript/zabbix.mailGraph.js
+
+Changes are in effect immediately, no need to restart any services.
+
+## mailGraph v2.10 release ##
+_(2023/06/30)_
+
+_This version has been verified with Zabbix 5.4, 6.0 LTS and 6.4, PHP 7.4 and 8.2 and recent versions of libraries used via composer._
+
+Minor updates to the mailGraph code
+- When not defining zabbix_api_user and zabbix_api_pwd in the configuration file the zbx_user and zbx_user_pwd wll be used
+
 ## Zabbix 6.4.x testing ##
+_(2023/06/30)_
+
+Zabbix 6.4 verification has succesfully completed.
+- Refactored code to remove deprecated and removed functions since Zabbix 6.4.0
+-- Zabbix Javascript now using HttpRequest instead of CurlHttpRequest (function name changes implemented)
+-- Zabbix API user.login is now based on "username" (instead of "user")
+
+Sidenotes
+- Zabbix logging still shows deprecation messages however it is believed these are internal to Zabbix and not related to mailGraph
+
 _(2023/06/29)_
 
 Zabbix 6.4.x verification is in progress. Required intermediate release to fix one major issue (Zabbix login parameters deprecation) and some minor coding updates.
 Expect to continue with automatic configuration within the next 2 months.
 
-## Zabbix 6.2.x testing ##
-_(2022/10/10)_
-
-Testing completed. No immediate issues found.
-
-_(2022/08/22)_
-
-Zabbix 6.2.x testing is in progress. Once completed the next effort will be to deliver on automatic configuration detection and automatic updates.
-
-## IMPORTANT NOTE ##
+## IMPORTANT NOTE for users of mailGraph and Zabbix versions under 5.4 ##
 As a result of a major functional change in Zabbix 5.4 (Screens no longer exist and are all moved into Dashboards) the mailGraph.screen macro no longer functions under Zabbix 5.4+. A code rewrite is in progress to deal with detecting the Zabbix version and to pick dashboard.get instead of screen.get as a source list for the graphs that should be included in the mail message.
 
 **v1.x is no longer supported; please upgrade to the current v2 release**
 **Please use the Wiki for information on how to install, configure and use MailGraph in Zabbix**
-
-## Example message
-The below message is just an example of what MailGraph is capable of. The template engine ("Twig") allows for a fully customized message creation to your needs! It is also possible to add more Zabbix fields. If you need additional fields just raise an issue ticket and ask and I'll see what I can do.
-
-[![](images/Example-mail-message-v122.png?raw=true)](images/Example-mail-message-v122.png)

javascript/READ.ME

@@ -1,7 +1 @@
-As per the details captured in issue #32 the javascript code portion of this Media Type has changed due to a platform/library change at Zabbix side.
-A distinction is now made for the following Zabbix versions:
-
-Zabbix <6.2 - use "zabbix.mailGraph.js"
-Zabbix >=6.2 - <coming soon>
-
-Please make sure you install the right javascript in your environment!
+Issue #32 has been resolved. The javascript code now works for Zabbix 5.4, 6.0 LTS and 6.4.

javascript/zabbix.mailGraph.js

@@ -1,7 +1,8 @@
+// mailGraph v2.11
 try {
     // Pickup parameters
     params = JSON.parse(value),
-        req = new CurlHttpRequest(),
+        req = new HttpRequest(),
         fields = {},
         resp = '',
         result = { tags: {} };
@@ -13,14 +14,26 @@ try {
     }
 
     // Declare output type
-    req.AddHeader('Content-Type: application/json');
+    req.addHeader('Content-Type: application/json');
 
     // Must have fields
-    fields.itemId = params.itemId;
-    fields.eventId = params.eventId;
+    fields.itemId = params.itemId * 1;
+    fields.eventId = params.eventId * 1;
     fields.recipient = params.recipient;
     fields.baseURL = params.baseURL;
-    fields.duration = params.duration;
+    fields.duration = params.duration * 1;
+
+    if (isNaN(fields.eventId)) {
+      throw '[MailGraph Webhook] Invalid event ID? Integer required (use actual event ID from Zabbix!)';
+    }
+
+    if (isNaN(fields.duration)) {
+      throw '[MailGraph Webhook] Invalid duration? Integer required (set to zero if unknown)!';
+    }
+
+    if (fields.recipient.charAt(0) == '{') {
+      throw '[MailGraph Webhook] Please define recipient for the test message!';
+    }
 
     // Optional fields
     if (typeof params.graphWidth === 'string') { fields.graphWidth = params.graphWidth; }
@@ -40,15 +53,27 @@ try {
 
     // Post information to the processing script
     Zabbix.Log(4, '[MailGraph Webhook] Sending request: ' + params.URL + '?' + JSON.stringify(fields));
-    var resp = req.Post(params.URL,JSON.stringify(fields));
-    Zabbix.Log(4, '[Mailgraph Webhook] Receiving response:' + resp);
+    var resp = req.post(params.URL,JSON.stringify(fields));
+    Zabbix.Log(4, '[Mailgraph Webhook] Received response:' + resp);
 
-    // If there was an error, report it
-    if (req.Status() != 200) { throw JSON.parse(resp).errors[0]; }
+    // If blank the email address is likely incorrect
+    if (resp==null) {
+      throw '[MailGraph Webhook] No data received from mailGraph! Likely the email processing failed? Check mailGraph logging';
+    }
+
+    // If there was an error, report it and stop
+    if (req.getStatus() != 200) { throw JSON.parse(resp).errors[0]; }
+
+    // If no datas returned, report it and stop
+    if (resp.charAt(0) == "!") {
+      throw '[MailGraph Webhook] No data received from mailGraph! Likely the event ID provided does not exist? Check mailGraph logging';
+    }
 
     // We expect the message id back from the processing script
-    resp = JSON.parse(resp);
-    result.tags.__message_id = resp.messageId;
+    msg = JSON.parse(resp);
+
+    result.tags.__message_id = msg.messageId;
+    Zabbix.Log(4, '[MailGraph Webhook] Message sent with identification "' + msg.messageId + '"');
 
     // Pass the result back to Zabbix
     return JSON.stringify(result);
@@ -56,6 +81,6 @@ try {
 catch (error)
 {
     // In case something went wrong in the processing, pass the error back to Zabbix
-    Zabbix.Log(127, 'MailGraph notification failed : '+error);
+    Zabbix.Log(127, 'MailGraph notification failed: '+error);
     throw 'MailGraph notification failed : '+error;
 }

mailGraph.php

@@ -8,6 +8,8 @@
     // upon an alert message.
     //
     // ------------------------------------------------------------------------------------------------------
+    // Release 1 tested with Zabbix 5.4 and 6.0 LTS
+    // ------------------------------------------------------------------------------------------------------
     // 1.00 2021/02/26 - Mark Oudsen - MVP version, ready for distribution
     // 1.01 2021/02/27 - Mark Oudsen - Enhanced search for associated graphs to an item // bug fixes
     // 1.10 2021/02/27 - Mark Oudsen - Moved all configuration outside code
@@ -40,10 +42,15 @@
     //      2021/07/05 - Mark Oudsen - Minor detail added: CLI debug mode now also outputs version
     //      2021/07/07 - Mark Oudsen - Fixed HTTPProxy typo in code (instead of HTTPPRoxy)
     // ------------------------------------------------------------------------------------------------------
+    // Release 2 tested with Zabbix 5.4, 6.0 LTS and 6.4 - tested with latest Composer libraries
+    // ------------------------------------------------------------------------------------------------------
     // 2.00 2021/12/16 - Mark Oudsen - itemId not always provisioned by Zabbix
     //                                 Several fixes on warning - several small bug fixes
     // 2.01 2021/12/16 - Mark Oudsen - Screens are no longer available - reverting to using Dashboards now
     // 2.02 2022/01/30 - Mark Oudsen - Added cleanup routine for old logs and images
+    // 2.10 2023/06/30 - Mark Oudsen - Refactored deprecated code - now compatible with Zabbix 6.0 LTS, 6.4
+    // 2.11 2023/07/01 - Mark Oudsen - Refactored Zabbix javascript - now capturing obvious errors
+    //                                 Added ability to locate latest problems for testing purposes
     // ------------------------------------------------------------------------------------------------------
     //
     // (C) M.J.Oudsen, mark.oudsen@puzzl.nl
@@ -65,7 +72,7 @@
     /////////////////////////////////////////////////////////////////////////////////////////////////////////
 
     // CONSTANTS
-    $cVersion = 'v2.020';
+    $cVersion = 'v2.11';
     $cCRLF = chr(10).chr(13);
     $maskDateTime = 'Y-m-d H:i:s';
     $maxGraphs = 4;
@@ -589,9 +596,20 @@
     $z_pass = $config['zabbix_user_pwd'];
 
     // Zabbix API user (requires Super Admin access rights)
+    // --- Copy from Zabbix user and override when defined in configuration
     // TODO: Check if information retreival can be done with less rigths
-    $z_api_user = $config['zabbix_api_user'];
-    $z_api_pass = $config['zabbix_api_pwd'];
+    $z_api_user = $z_user;
+    $z_api_pass = $z_pass;
+
+    if (isset($config['zabbix_api_user']))
+    {
+        $z_api_user = $config['zabbix_api_user'];
+    }
+
+    if (isset($config['zabbix_api_pwd']))
+    {
+        $z_api_pass = $config['zabbix_api_pwd'];
+    }
 
     // Mail sender
     $mailFrom = array($config['mail_from']=>'Zabbix Mailgraph');
@@ -627,7 +645,7 @@
 
     $request = array('jsonrpc'=>'2.0',
                      'method'=>'user.login',
-                     'params'=>array('user'=>$z_api_user,
+                     'params'=>array('username'=>$z_api_user,
                                      'password'=>$z_api_pass),
                      'id'=>nextRequestID(),
                      'auth'=>null);
@@ -647,6 +665,31 @@
 
     _log('> Token = '.$token);
 
+    // -----------------------------------
+    // --- IF NO EVENT ID FETCH LATEST ---
+    // -----------------------------------
+
+    if ($p_eventId=="0")
+    {
+        _log('# No event ID given, picking up random event from Zabbix');
+
+        $request = array('jsonrpc'=>'2.0',
+                         'method'=>'problem.get',
+                         'params'=>array('output'=>'extend',
+                                         'recent'=>'true',
+                                         'limit'=>1),
+                         'auth'=>$token,
+                         'id'=>nextRequestID());
+
+        $thisProblems = postJSON($z_url_api, $request);
+        _log('> Problem data'.$cCRLF.json_encode($thisProblems,JSON_PRETTY_PRINT|JSON_NUMERIC_CHECK));
+
+        if (!isset($thisProblems['result'][0])) { echo '! No response data received?'.$cCRLF; die; }
+
+        $p_eventId = $thisProblems['result'][0]['eventid'];
+        _log('> Picked up random last event #'.$p_eventId);
+    }
+
     // ------------------------------
     // --- READ EVENT INFORMATION ---
     // ------------------------------
@@ -1424,8 +1467,15 @@
 
     // Return Event TAG information for Zabbix
 
-    $response = array('messageId.mailGraph'=>$messageId);
-    echo json_encode($response).$cCRLF;
+    $response = array('messageId'=>$messageId);
+
+    if ($response=="")
+    {
+        _log("! Failed to send mail message");
+        echo "! Failed to send mail message. Likely an issue with the recipient email address?".$cCRLF;
+    } else {
+        echo json_encode($response).$cCRLF;
+    }
 
     /////////////////////////////////////////////////////////////////////////////////////////////////////////
     // Wrap up //////////////////////////////////////////////////////////////////////////////////////////////

mailGraph.xml

@@ -1,7 +1,7 @@
 <?xml version="1.0" encoding="UTF-8"?>
 <zabbix_export>
-    <version>5.0</version>
-    <date>2021-03-17T12:55:43Z</date>
+    <version>5.4</version>
+    <date>2023-07-02T10:34:30Z</date>
     <media_types>
         <media_type>
             <name>MailGraph</name>
@@ -9,7 +9,7 @@
             <parameters>
                 <parameter>
                     <name>baseURL</name>
-                    <value>https://mydomain.com/zabbix/</value>
+                    <value>https://myzabbix.example.com/</value>
                 </parameter>
                 <parameter>
                     <name>duration</name>
@@ -61,127 +61,152 @@
                 </parameter>
                 <parameter>
                     <name>URL</name>
-                    <value>https://mydomain.com/mailGraph.php</value>
+                    <value>https://myzabbix.example.com/mailGraph.php</value>
                 </parameter>
             </parameters>
-            <script>try {&#13;
-    // Pickup parameters&#13;
-    params = JSON.parse(value),&#13;
-        req = new CurlHttpRequest(),&#13;
-        fields = {},&#13;
-        resp = '',&#13;
-        result = { tags: {} };&#13;
-&#13;
-    // Set HTTP proxy if required&#13;
-    if (typeof params.HTTPProxy === 'string' &amp;&amp; params.HTTPProxy.trim() !== '') {&#13;
-        req.setProxy(params.HTTPProxy);&#13;
-        fields.HTTPProxy = params.HTTPProxy;&#13;
-    }&#13;
-&#13;
-    // Declare output type&#13;
-    req.AddHeader('Content-Type: application/json');&#13;
-&#13;
-    // Must have fields&#13;
-    fields.itemId = params.itemId;&#13;
-    fields.eventId = params.eventId;&#13;
-    fields.recipient = params.recipient;&#13;
-    fields.baseURL = params.baseURL;&#13;
-    fields.duration = params.duration;&#13;
-&#13;
-    // Optional fields&#13;
-    if (typeof params.graphWidth === 'string') { fields.graphWidth = params.graphWidth; }&#13;
-    if (typeof params.graphHeight === 'string') { fields.graphHeight = params.graphHeight; }&#13;
-    if (typeof params.subject === 'string') { fields.subject = params.subject; }&#13;
-    if (typeof params.showLegend === 'string') { fields.showLegend = params.showLegend; }&#13;
-    if (typeof params.periods === 'string') { fields.periods = params.periods; }&#13;
-    if (typeof params.periods_headers === 'string') { fields.periods_headers = params.periods_headers; }&#13;
-    if (typeof params.debug === 'string') { fields.debug = params.debug; }&#13;
-&#13;
-    // Add generic fields&#13;
-    Object.keys(params).forEach(function(key) {&#13;
-        if (key.substring(0, 4) == 'info') {&#13;
-            fields[key] = params[key];&#13;
-        }&#13;
-    });&#13;
-&#13;
-    // Post information to the processing script&#13;
-    Zabbix.Log(4, '[MailGraph Webhook] Sending request: ' + params.URL + '?' + JSON.stringify(fields));&#13;
-    var resp = req.Post(params.URL,JSON.stringify(fields));&#13;
-    Zabbix.Log(4, '[Mailgraph Webhook] Receiving response:' + resp);&#13;
-&#13;
-    // If there was an error, report it&#13;
-    if (req.Status() != 200) { throw JSON.parse(resp).errors[0]; }&#13;
-&#13;
-    // We expect the message id back from the processing script&#13;
-    resp = JSON.parse(resp);&#13;
-    result.tags.__message_id = resp.messageId;&#13;
-&#13;
-    // Pass the result back to Zabbix&#13;
-    return JSON.stringify(result);&#13;
-}&#13;
-catch (error)&#13;
-{&#13;
-    // In case something went wrong in the processing, pass the error back to Zabbix&#13;
-    Zabbix.Log(127, 'MailGraph notification failed : '+error);&#13;
-    throw 'MailGraph notification failed : '+error;&#13;
+            <script>// mailGraph v2.11
+try {
+    // Pickup parameters
+    params = JSON.parse(value),
+        req = new HttpRequest(),
+        fields = {},
+        resp = '',
+        result = { tags: {} };
+
+    // Set HTTP proxy if required
+    if (typeof params.HTTPProxy === 'string' &amp;&amp; params.HTTPProxy.trim() !== '') {
+        req.setProxy(params.HTTPProxy);
+        fields.HTTPProxy = params.HTTPProxy;
+    }
+
+    // Declare output type
+    req.addHeader('Content-Type: application/json');
+
+    // Must have fields
+    fields.itemId = params.itemId * 1;
+    fields.eventId = params.eventId * 1;
+    fields.recipient = params.recipient;
+    fields.baseURL = params.baseURL;
+    fields.duration = params.duration * 1;
+
+    if (isNaN(fields.eventId)) {
+      throw '[MailGraph Webhook] Invalid event ID? Integer required (use actual event ID from Zabbix!)';
+    }
+
+    if (isNaN(fields.duration)) {
+      throw '[MailGraph Webhook] Invalid duration? Integer required (set to zero if unknown)!';
+    }
+
+    if (fields.recipient.charAt(0) == '{') {
+      throw '[MailGraph Webhook] Please define recipient for the test message!';
+    }
+
+    // Optional fields
+    if (typeof params.graphWidth === 'string') { fields.graphWidth = params.graphWidth; }
+    if (typeof params.graphHeight === 'string') { fields.graphHeight = params.graphHeight; }
+    if (typeof params.subject === 'string') { fields.subject = params.subject; }
+    if (typeof params.showLegend === 'string') { fields.showLegend = params.showLegend; }
+    if (typeof params.periods === 'string') { fields.periods = params.periods; }
+    if (typeof params.periods_headers === 'string') { fields.periods_headers = params.periods_headers; }
+    if (typeof params.debug === 'string') { fields.debug = params.debug; }
+
+    // Add generic fields
+    Object.keys(params).forEach(function(key) {
+        if (key.substring(0, 4) == 'info') {
+            fields[key] = params[key];
+        }
+    });
+
+    // Post information to the processing script
+    Zabbix.Log(4, '[MailGraph Webhook] Sending request: ' + params.URL + '?' + JSON.stringify(fields));
+    var resp = req.post(params.URL,JSON.stringify(fields));
+    Zabbix.Log(4, '[Mailgraph Webhook] Received response:' + resp);
+
+    // If blank the email address is likely incorrect
+    if (resp==null) {
+      throw '[MailGraph Webhook] No data received from mailGraph! Likely the email processing failed? Check mailGraph logging';
+    }
+
+    // If there was an error, report it and stop
+    if (req.getStatus() != 200) { throw JSON.parse(resp).errors[0]; }
+
+    // If no datas returned, report it and stop
+    if (resp.charAt(0) == &quot;!&quot;) {
+      throw '[MailGraph Webhook] No data received from mailGraph! Likely the event ID provided does not exist? Check mailGraph logging';
+    }
+
+    // We expect the message id back from the processing script
+    msg = JSON.parse(resp);
+
+    result.tags.__message_id = msg.messageId;
+    Zabbix.Log(4, '[MailGraph Webhook] Message sent with identification &quot;' + msg.messageId + '&quot;');
+
+    // Pass the result back to Zabbix
+    return JSON.stringify(result);
+}
+catch (error)
+{
+    // In case something went wrong in the processing, pass the error back to Zabbix
+    Zabbix.Log(127, 'MailGraph notification failed: '+error);
+    throw 'MailGraph notification failed : '+error;
 }</script>
             <process_tags>YES</process_tags>
-            <description>The &quot;URL&quot; must point to the location of the processing script. If a proxy is required, define &quot;HTTPProxy&quot; for the proxy address.&#13;
-&#13;
-Customization:&#13;
-- &quot;graphWidth&quot; and &quot;graphWidth&quot; can be defined for the image size&#13;
-- &quot;showLegend&quot; can be defined to show or hide the legend of the graph&#13;
-- &quot;subject&quot; can be defined for a customized subject for the mail message&#13;
-- &quot;periods&quot; and &quot;periods_headers&quot; can be defined for displaying multiple periods of the same graph, or&#13;
-- &quot;period&quot; and &quot;period_header&quot; can be defined to display a single graph&#13;
-&#13;
-The html.template and plain.template files can be adjusted (TWIG format).&#13;
-&#13;
+            <description>The &quot;URL&quot; must point to the location of the processing script. If a proxy is required, define &quot;HTTPProxy&quot; for the proxy address.
+
+Customization:
+- &quot;graphWidth&quot; and &quot;graphWidth&quot; can be defined for the image size
+- &quot;showLegend&quot; can be defined to show or hide the legend of the graph
+- &quot;subject&quot; can be defined for a customized subject for the mail message
+- &quot;periods&quot; and &quot;periods_headers&quot; can be defined for displaying multiple periods of the same graph, or
+- &quot;period&quot; and &quot;period_header&quot; can be defined to display a single graph
+
+The html.template and plain.template files can be adjusted (TWIG format).
+
 More details are available at https://github.com/moudsen/mailGraph</description>
             <message_templates>
                 <message_template>
                     <event_source>TRIGGERS</event_source>
                     <operation_mode>PROBLEM</operation_mode>
                     <subject>Problem: {EVENT.NAME}</subject>
-                    <message>Problem started at {EVENT.TIME} on {EVENT.DATE}&#13;
-Problem name: {EVENT.NAME}&#13;
-Host: {HOST.NAME}&#13;
-Severity: {EVENT.SEVERITY}&#13;
-Operational data: {EVENT.OPDATA}&#13;
-Original problem ID: {EVENT.ID}&#13;
-{TRIGGER.URL}&#13;
-&#13;
-eventId: {EVENT.ID}&#13;
-TriggerId: {TRIGGER.ID}&#13;
+                    <message>Problem started at {EVENT.TIME} on {EVENT.DATE}
+Problem name: {EVENT.NAME}
+Host: {HOST.NAME}
+Severity: {EVENT.SEVERITY}
+Operational data: {EVENT.OPDATA}
+Original problem ID: {EVENT.ID}
+{TRIGGER.URL}
+
+eventId: {EVENT.ID}
+TriggerId: {TRIGGER.ID}
 itemId: {ITEM.ID]</message>
                 </message_template>
                 <message_template>
                     <event_source>TRIGGERS</event_source>
                     <operation_mode>RECOVERY</operation_mode>
-                    <subject>Resolved in {EVENT.DURATION}: {EVENT.NAME}</subject>
-                    <message>Problem has been resolved at {EVENT.RECOVERY.TIME} on {EVENT.RECOVERY.DATE}&#13;
-Problem name: {EVENT.NAME}&#13;
-Problem duration: {EVENT.DURATION}&#13;
-Host: {HOST.NAME}&#13;
-Severity: {EVENT.SEVERITY}&#13;
-Original problem ID: {EVENT.ID}&#13;
-{TRIGGER.URL}&#13;
-&#13;
-eventId: {EVENT.ID}&#13;
-TriggerId: {TRIGGER.ID}&#13;
+                    <subject>Resolved in {EVENT.DURATION}: {EVENT.RECOVERY.NAME}</subject>
+                    <message>Problem has been resolved at {EVENT.RECOVERY.TIME} on {EVENT.RECOVERY.DATE}
+Problem name: {EVENT.RECOVERY.NAME}
+Problem duration: {EVENT.DURATION}
+Host: {HOST.NAME}
+Severity: {EVENT.SEVERITY}
+Original problem ID: {EVENT.ID}
+{TRIGGER.URL}
+
+eventId: {EVENT.ID}
+TriggerId: {TRIGGER.ID}
 itemId: {ITEM.ID]</message>
                 </message_template>
                 <message_template>
                     <event_source>TRIGGERS</event_source>
                     <operation_mode>UPDATE</operation_mode>
                     <subject>Updated problem in {EVENT.AGE}: {EVENT.NAME}</subject>
-                    <message>{USER.FULLNAME} {EVENT.UPDATE.ACTION} problem at {EVENT.UPDATE.DATE} {EVENT.UPDATE.TIME}.&#13;
-{EVENT.UPDATE.MESSAGE}&#13;
-&#13;
-Current problem status is {EVENT.STATUS}, age is {EVENT.AGE}, acknowledged: {EVENT.ACK.STATUS}.&#13;
-&#13;
-eventId: {EVENT.ID}&#13;
-TriggerId: {TRIGGER.ID}&#13;
+                    <message>{USER.FULLNAME} {EVENT.UPDATE.ACTION} problem at {EVENT.UPDATE.DATE} {EVENT.UPDATE.TIME}.
+{EVENT.UPDATE.MESSAGE}
+
+Current problem status is {EVENT.STATUS}, age is {EVENT.AGE}, acknowledged: {EVENT.ACK.STATUS}.
+
+eventId: {EVENT.ID}
+TriggerId: {TRIGGER.ID}
 itemId: {ITEM.ID]</message>
                 </message_template>
             </message_templates>