How to make a SOAP call through curl

This article explains the way to invoke a SOAP service through curl command and this may be useful when isolationg the issues in an integration environment.

 curl -ivs -H "Content-Type: text/xml; charset=utf-8" -d @test.xml -w "@performance-format.txt" -X POST 'http://192.168.1.5/test.asmx'   

Here, test.xml will be the message body of the request.

 <soapenv:Envelope xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/">  
   <soapenv:Header/>  
   <soapenv:Body>  
    <test/>  
   </soapenv:Body>  
 </soapenv:Envelope>  

performance-format.txt will be as below, this is to find out the request-response time.

 -----------------------------------------\n  
 url_effective: %{url_effective}\n   
 http_code: %{http_code}\n   
 content_type: %{content_type}\n   
 time_namelookup: %{time_namelookup}\n   
 time_connect: %{time_connect}\n   
 time_appconnect: %{time_appconnect}\n   
 time_pretransfer: %{time_pretransfer}\n   
 time_redirect: %{time_redirect}\n   
 time_starttransfer: %{time_starttransfer}\n   
 ----------------------------------------\n   
 time_total: %{time_total}\n   
 ----------------------------------------\n   

For more information on performance-format please refer [1].

[1] http://ajanthane.blogspot.com/2017/08/calculating-request-and-response-time.html

Usage of YYYY and yyyy when retrieving the year in a last week of a Calendar Year with SYSTEM_DATE property in WSO2 ESB 5.0.0

Usage of YYYY and yyyy when retrieving the year in a last week of a Calendar Year with SYSTEM_DATE property in WSO2 ESB 5.0.0

This article explains the difference between year retrieved using the below two properties in WSO2 ESB.

1) <property name="YEAR" expression="get-property('SYSTEM_DATE', 'YYYY')"/>
   
2) <property name="year" expression="get-property('SYSTEM_DATE', 'yyyy')"/>

The sample proxy service used is as below:

 <?xml version="1.0" encoding="UTF-8"?>  
 <proxy xmlns="http://ws.apache.org/ns/synapse"  
     name="WeekYearTestProxy"  
     transports="https http"  
     startOnLoad="true">  
   <description/>  
   <target>  
    <inSequence>  
      <log level="custom">  
       <property name="YEAR" expression="get-property('SYSTEM_DATE', 'YYYY')"/>  
       <property name="year" expression="get-property('SYSTEM_DATE', 'yyyy')"/>  
      </log>  
    </inSequence>  
   </target>  
 </proxy>  

The output log of the above proxy will be as below on 31/12/2017.

When you use YYYY as the format, then you may get into an issue during a last week of year as for example in my case it was 31/12/2017. When this property retrieve the year, it will print it as 2018. Why this happens is because YYYY will return the Weekly based calendar year and how that is calculated is as below and refer [1] for more information.

[1] https://docs.oracle.com/javase/8/docs/api/java/util/GregorianCalendar.html#week_year

Consider, I'm running the sample proxy on 31/12/2017 and it is a Sunday. Based on [1], we need to check the what value set to FirstDayOfWeek and the other parameter we need to check is the MinimalDaysInFirstWeek. We can check both of this using the below sample code:

 import java.util.Calendar;  
 import java.util.GregorianCalendar;  
 import java.util.Locale;  
 public class Test{  
      public static void main(String[] args) {  
           Calendar c = new GregorianCalendar();  
           System.out.println(Locale.getDefault() + ": " + c.getFirstDayOfWeek() + " - " + c.getMinimalDaysInFirstWeek());  
      }  
 }  

Output of the above Java code segment will be as below:

Above output tells us that the first day of week is Sunday and Minimal days in first week is 1.

Below are extracted from [2] [3] to get more understanding on the methods used above.
[2] https://docs.oracle.com/javase/8/docs/api/java/util/Calendar.html#getMinimalDaysInFirstWeek--

getMinimalDaysInFirstWeek

Gets what the minimal days required in the first week of the year are; e.g., if the first week is defined as one that contains the first day of the first month of a year, this method returns 1. If the minimal days required must be a full week, this method returns 7.

[3] https://docs.oracle.com/javase/8/docs/api/java/util/Calendar.html#getFirstDayOfWeek--

getFirstDayOfWeek

Gets what the first day of the week is; e.g., SUNDAY in the U.S., MONDAY in France.

Based on the above definitions and our output.
FirstDayofWeek is Sunday and as getMinimalDaysInFirstWeek returns as 1, the first week is the one that contains the first day of the first month of the year. Due to that the first week starts from Sunday 31/12/2017, due to that when we use YYYY, the year that will be returned is 2018.

That's how the YYYY, get's 2018...

For more info refer below section in [1].

Splitting WSO2 wire enabled logs based on size and backup them to another folder

This article explains how we can split the log files based on size and also how we can move the backup files to another folder using shell script. When we use log4j configuration to split the file based on size, we have a limitation that there is a possibility that we may lose a particular backup file, as it works as below. To avoid that we need to create a backup script and that also provided in this article.

Consider we are setting as below:
log4j.appender.CARBON_LOGFILE.MaxFileSize=1MB
log4j.appender.CARBON_LOGFILE.MaxBackupIndex=5

It will keep only 5 backup files, when the 6th ( means the wso2carbon.log reached 1MB ), it will delete the 1st backup file created that will be wso2carbon.log.5 and reassign the order, like wise old ones will be deleted keeping the latest.

Below are the steps to follow:

1) Take a backup of the ESB_HOME/repository/conf/log4j.properties.
2) Enable the wire logs in log4j.properties

Uncomment the below lines:

 #log4j.logger.org.apache.synapse.transport.http.headers=DEBUG  
 #log4j.logger.org.apache.synapse.transport.http.wire=DEBUG  

3) Follow the instructions [1].
[1] https://docs.wso2.com/display/ADMIN44x/Monitoring+Logs
Change the log4j.appender.CARBON_LOGFILE=org.wso2.carbon.utils.logging.appenders.CarbonDailyRollingFileAppenderappender
in the <PRODUCT_HOME>/repository/conf/ log4j.properties file as follows:

 log4j.appender.CARBON_LOGFILE=org.apache.log4j.RollingFileAppender  

Add the following two properties under RollingFileAppender

 log4j.appender.CARBON_LOGFILE.MaxFileSize=100MB   
 log4j.appender.CARBON_LOGFILE.MaxBackupIndex=200  

4) Restart the Server.

Once done you can see the output inside ESB_HOME/repository/logs folder as below:

5) Configure the cron job to run the backup script.

 cd /etc/cron.daily/  
 chmod 755 wso2backuplogscript-esb.sh  

 #!/bin/bash  
 ESB_HOME="/home/ajanthan/log_split/wso2esb-5.0.0"  
 DIRECTORY="$ESB_HOME/repository/logs"  
 BACKUP_SRC_FILES="wso2carbon.log.*"  
 DESTINATION="$DIRECTORY/archivefolder"  
 TIME=`date "+%Y%m%d-%H%M%S"`  
 FILENAME=backup-wso2carbon.$TIME.tar.gz  
 echo "[`date '+%Y-%m-%d %H:%M:%S'`] Starting the backup process...\r\n" ;  
 echo "Source Directory : $DIRECTORY\n";  
 echo "Destination Directory : $DESTINATION\n";  
 if ls $DIRECTORY/$BACKUP_SRC_FILES 1> /dev/null 2>&1;  
     then  
         echo "[`date '+%Y-%m-%d %H:%M:%S'`] Files available for backup. \n";  
         cd $DIRECTORY && tar -cvpzf $DESTINATION/$FILENAME $BACKUP_SRC_FILES #create,verbose,preserve permission,zip,filename  
     if [ -f "$DESTINATION/$FILENAME" ]  
         then  
         #echo -e "Inside then...\n";  
         echo "[`date '+%Y-%m-%d %H:%M:%S'`] $DESTINATION/$FILENAME found.\n";  
         echo "[`date '+%Y-%m-%d %H:%M:%S'`] Starting to remove the files from Source : $BACKUP_SRC_FILES\r\n";  
         rm -fv $BACKUP_SRC_FILES;  #Avoiding the prompt for each file and also printing what are the files got removed by using -fv.  
         echo "[`date '+%Y-%m-%d %H:%M:%S'`] File removal END.\r\n";  
     else  
         #echo -e "Inside elsee...\n";  
         echo "[`date '+%Y-%m-%d %H:%M:%S'`] $DESTINATION/$FILENAME not found.\r\n";  
     fi  
 else  
     echo "[`date '+%Y-%m-%d %H:%M:%S'`] Files do not available for backup.\n";  
 fi  
 echo "\r\n[`date '+%Y-%m-%d %H:%M:%S'`] Ending the backup process...\r\n";  

6) Create a folder for the backup files

ESB_HOME/repository/logs/archivefolder
ESB_HOME/repository/logs/backupprocesslogs

7) Add new cron job to crontab:

crontab -e

For Testing

*/5 * * * * sh /etc/cron.daily/wso2backuplogscript-esb.sh >> ESB_HOME/repository/logs/backupprocesslogs/backup-$(date +\%Y-\%m-\%d-\%H-\%M-\%S).log 2>&1


8) Check the log at $ESB_HOME/repository/logs/backupprocesslogs-esb/
Check the backup files at $ESB_HOME/repository/logs/archive-esblogs

Also can check the process related log at ESB_HOME/repository/logs/backupprocesslogs. The sample output will be as below: