Version 4.1.0.

This is the first production release in the v4 series (v4.1).  
It is regarded as stable and only bug-fixes releases will be
released in this series.

Note there will be significant changes to the configuration 
files in later releases (v4.2 on).

The new version is written in c++11 and so to compile it you will need gcc v5.4 
or later. (or another complier that supports the c++11 std::thread library).  
Note that the target systems may also need an c++11 library update.

REVISED PROCESS MODEL

The parent children process model (which does not scale for very large numbers 
of connections) is replaced with a queue/threads based model.

The main thread now only deals with set-up of the logging, listener, and worker 
threads, the input (and reinput) of the lists, signals and statistics.  

The treads communicate via fi-fo queues within memory and so there is no need 
for ipc pipes.

A listener thread is set up for each ip/port combination. They listen for a 
connection, accept it and then push the new connection socket on the 
appropriate worker queue.

The worker threads pop connections from the worker queue and deal with the 
connection.

When a worker wants to log a request it pushes the logging data onto the log 
queue. The logging thread will pop the data from the queue, format it and 
write it to the log. 

Most of the above logic is in FatControler.cpp.  The logic is now much simpler 
and has reduced the amount of code in FatControler by over 50%.

Socket classes have been extensively modified to remove interrupt handling (for 
list reload etc) and all select calls are removed.  So there is no longer a 
need to modify FD_SETSIZE.

New LOptionContainer class has been written to hold list and filter group 
setings.   On gentle restart a new LOptionContainer object is created and 
loaded with filter group and list settings.   Once fully read in a global 
shared pointer is switched from the old list to the new, making actioning 
list changes immediate an with no interruption to service.

TESTING -  basic and (scripted) load testing with over 10000 worker threads 
has been carried out to date.  Showng a big reduction in memory 
usage. 

It also handles transient peaks more smoothly.  Connections are queued, so 
response will slow down, but not stop when all the worker threads are busy.

TODO - before first v4 stable release - PLEASE HELP

  Testing on various platforms.  V4 is using c++11, so should work on any
  platform with c++11 support.  The target platforms to be supported are
  Linux and FreeBsd.  Can you help with this?

  FreeBsd testing - should compile with c++11 in FreeBsd - and now work
  with more than 1024 worker threads. NEEDS compiling and testing by 
  FreeBsd community.

  Testing of NTLM, proxy-basic, proxy-digest and ident authplugins

TODO - for post v4.1 releases.

Make lists parameter based rather than hard coded.

Create 'story-board' parameterised logic for handling filtering logic.

Transparent https option.

Re-implement mailing option.

WISHLIST

ICAP server mode.

Per-user exceptions/blocks.

NOTES  FOR THIS VERSION  - v4.1.0

All pics support has been removed

Mail option not yet implemented.

Url cache not implemented

IP cache not implemented

Auth plugins - IP, Port, DNSauth, proxy-basic and proxy-digest tested 
and working

Scan plugins - test status not clear

Configuration in this version is largely backward compatible with v3.4
just two new options in e2guardian.conf

   httpworkers
   enablessl

Also monitor_flag and monitor_helper options are no longer triggered by 
upstream cache failure, but just by starting and stopping e2guardian.  
Upstream cache failure can now be detected by other means  - e.g. user error 
message.


Please note the new format for the dstats log. See notes/dstat_format

Philip Pearce 13 April 2017


