logger.props - Console MySQL and File Output for Rule Eval

Contains tips for configurators working with Aware IM
Post Reply
PointsWell
Posts: 1457
Joined: Tue Jan 24, 2017 5:51 am
Location: 'Stralya

logger.props - Console MySQL and File Output for Rule Eval

Post by PointsWell »

So after a bit of trial and error with log4j and a bit of help from here I have managed to make my app log in a way that I can read and in a database instead of a log file, capturing the BSV and user.
Screen Shot 2017-07-25 at 16.08.38.png
Screen Shot 2017-07-25 at 16.08.38.png (64.82 KiB) Viewed 21115 times
And this is the file output

Code: Select all

2017-07-25, 06:07:03.592, 'Test Logger, Thread-96, GENERIC, Map#Admin, 17  
2017-07-25, 06:07:03.599, 'Test Logger, Thread-96, GENERIC, Map#Admin, 17  
2017-07-25, 06:07:03.604, 'Test Logger, Thread-96, GENERIC, Map#Admin, 20 Executing process ShowPerson
2017-07-25, 06:07:03.610, 'Test Logger, Thread-96, ACTION_EXECUTION, Map#Admin   , 34 Value of Person could not be resolved
2017-07-25, 06:07:03.612, 'Test Logger, Thread-96, RULE_EVALUATION, Map#Admin   , 29 Started evaluation of rules
2017-07-25, 06:07:03.617, 'Test Logger, Thread-96, RULE_EVALUATION, Map#Admin      , 22 Evaluating rule 'Show' CREATE Person EDIT Person USING Main
2017-07-25, 06:07:03.619, 'Test Logger, Thread-96, RULE_EVALUATION, Map#Admin         , 2 Adding action CREATE Person to the agenda
2017-07-25, 06:07:03.620, 'Test Logger, Thread-96, RULE_EVALUATION, Map#Admin         , 2 Adding action EDIT Person USING Main to the agenda
2017-07-25, 06:07:03.621, 'Test Logger, Thread-96, RULE_EVALUATION, Map#Admin      , 16 Finished rule evaluation
2017-07-25, 06:07:03.621, 'Test Logger, Thread-96, RULE_EVALUATION, Map#Admin   , 16 Finished evaluation of rules
2017-07-25, 06:07:03.622, 'Test Logger, Thread-96, ACTION_EXECUTION, Map#Admin   , 15 Executing action CREATE Person
2017-07-25, 06:07:03.672, 'Test Logger, Thread-96, ACTION_EXECUTION, Map#Admin   , 16 Finished executing action CREATE Person
2017-07-25, 06:07:03.674, 'Test Logger, Thread-96, ACTION_EXECUTION, Map#Admin   , 15 Executing action EDIT Person USING Main
2017-07-25, 06:07:03.675, 'Test Logger, Thread-96, ACTION_EXECUTION, Map#Admin   , 16 Finished executing action EDIT Person USING Main
2017-07-25, 06:07:03.682, 'Test Logger, Thread-96, GENERIC, Map#Admin, 21 Process ShowPerson is suspended. Its id is 3
2017-07-25, 06:07:03.686, 'Test Logger, Thread-96, GENERIC, Map#Admin, 17  
2017-07-25, 06:07:03.691, 'Test Logger, Thread-96, GENERIC, Map#Admin, 17  
Attached is a logger.props file with how I did it.

The requirements are:
  • Database - I called mine logging
  • 3 tables
    • log_general
    • log_business
    • log_test
I parameterised everything as much as possible.

Update the parameters with
  • Path to your database
  • User name
  • Password
  • update the logdir to wherever you want your logger files to go. Be aware that the files are all being placed in related subdirectories of this logdir, if you don't want this to happen you will need to edit everywhere that files are created
If you want to add more logging information then you need to update your table to add extra columns and then add the % value in the insert.

I have left everything to continue to log to files, though I have changed these files to be RollingDaily. This means at your server's midnight it will roll over to a new file and rename the old one.

I have belt and braced the date to include, date, time and a database timestamp. You could probably reduce this but I figured that it is easier for filtering to have all three.

The why: by putting my logs into MySQL I can monitor rule execution on a remote server (log viewer isn't available as I don't have x-server running) in a semi real time, whereas with just a file I have to use an FTP retrieve file or open in an ssh session and then view the file and obviously the file I am viewing will not update

-------------------------------------------------------------------------
Standard disclaimer:
This probably won't ruin you financially, but if it does, you didn't get it from me.
If your pets suddenly don't love you, your milk goes off, or your hot water tank busts, I disclaim all responsibility.
Last edited by PointsWell on Fri Jul 28, 2017 3:59 am, edited 2 times in total.
PointsWell
Posts: 1457
Joined: Tue Jan 24, 2017 5:51 am
Location: 'Stralya

Re: Better Logging By mySQL

Post by PointsWell »

Updated logger.props to:
  • correct mismatch between system logger definition and usage
  • remove trailing quote mark in a logger name
  • copy paste error in one of the loggers
  • updated the comments
  • added different roll patterns for files - previously was all or nothing
  • mismatch between syslog parameter definition and usage
logger.props updated - see subsequent post.
Last edited by PointsWell on Fri Jul 28, 2017 3:57 am, edited 1 time in total.
PointsWell
Posts: 1457
Joined: Tue Jan 24, 2017 5:51 am
Location: 'Stralya

Re: Better Logging By mySQL

Post by PointsWell »

There is a known issue with single quotes being the only way to get this to work, so the logger with throw an exception error if it tries to log any reserved words where they follow a string

e.g. 'Some text' INSERT

Use corrected file below.
Last edited by PointsWell on Sat Jul 29, 2017 10:11 am, edited 1 time in total.
hpl123
Posts: 2579
Joined: Fri Feb 01, 2013 1:13 pm
Location: Scandinavia

Re: logger.props - Console MySQL and File Output for Rule Ev

Post by hpl123 »

Very nice tip and I have been wanting to set this up myself for ages, thanks for sharing :D.
Henrik (V8 Developer Ed. - Windows)
tford
Posts: 4238
Joined: Sat Mar 10, 2007 6:44 pm

Re: logger.props - Console MySQL and File Output for Rule Ev

Post by tford »

Nice work!
Tom - V8.8 build 3137 - MySql / PostGres
Jaymer
Posts: 2430
Joined: Tue Jan 13, 2015 10:58 am
Location: Tampa, FL
Contact:

Re: logger.props - Console MySQL and File Output for Rule Ev

Post by Jaymer »

I understand "the why", but I'm missing the bigger picture.

is "log4j" the system that AIM uses now?
... and you've modified the way it works to go to a file?

Or is this [just] a framework for me to log activities in my application?

To use it, I call a process (your routine) during my processes and it writes an entry?

thx
jaymer...
Click Here to see a collection of my tips & hacks on this forum. Or search for "JaymerTip" in the search bar at the top.

Jaymer
Aware Programming & Consulting - Tampa FL
PointsWell
Posts: 1457
Joined: Tue Jan 24, 2017 5:51 am
Location: 'Stralya

Re: logger.props - Console MySQL and File Output for Rule Ev

Post by PointsWell »

Jaymer wrote:I understand "the why", but I'm missing the bigger picture.

is "log4j" the system that AIM uses now?
... and you've modified the way it works to go to a file?

Or is this [just] a framework for me to log activities in my application?

To use it, I call a process (your routine) during my processes and it writes an entry?

thx
jaymer...
Good questions.

Log4j is the framework that AIM uses - so it is all existing functionality, this file is just the properties that drive the log4j jar.. There is already a logger.props file in /Awareim/bin so when you download this one, rename the old one to logger.props.old or something then copy he downloaded version into the bin directory

There's nothing else for you to do other than create a database for your logger and update the user (for your logging database)password (for the user), URL (to the database) and dirlog parameters then restart AIM. At the moment it only has the driver details for MySQL so if you are using Sqlserver or Oracle you'll need to update that too. It does not log to Derby - I couldn't find a decent Derby GUI to make it worthwhile logging to Derby and I don't use Derby.

The changes I made have been to tidy up the format of the logger file, parameterise the whole thing (to make it simpler (for me at least), add the ability to log to a database and change the roll over method for the files that are being written to be replaced each day, the default was to produce files limited by size where they were limited at all.
PointsWell
Posts: 1457
Joined: Tue Jan 24, 2017 5:51 am
Location: 'Stralya

Re: Better Logging By mySQL

Post by PointsWell »

PointsWell wrote:There is a known issue with single quotes being the only way to get this to work, so the logger with throw an exception error if it tries to log any reserved words where they follow a string

e.g. 'Some text' INSERT

Use corrected file below.
Attachments
logger.props 2.zip
(3.54 KiB) Downloaded 929 times
PointsWell
Posts: 1457
Joined: Tue Jan 24, 2017 5:51 am
Location: 'Stralya

'Interesting' Behaviour

Post by PointsWell »

Hmm I have found an 'interesting' behaviour with the Database logger.

It seems that when it trips over to the next day it forgets to log to the database. The file loggers still work though

The recovery is to restart AwareIM - These are not party manners.

Oh well it works -ish but not brilliantly and not all the time.

In my research there is a newer version of log4j that is more robust, speaking with Support it is not of high priority to replace and is a pain to do.

log4j has a companion jar Apache Extras https://logging.apache.org/log4j/extras ... mmary.html which enhances the use of DB logging via DBAppender but this is well beyond my capabilities and skill.

So my original warning / get out seems pertinent now, it works kinda but not always - though still helpful if you are working with a remote server and have no GUI.
Post Reply