CentOS 7 permanent systemd log

One of the most annoying thing about CentOS 7 (actually, one of the very few annoying thing!) is the default configuration involving systemd journal logs.

The default is indeed to clear the whole log on every boot.

If you’d like to preserve the journal (to being able to see what happened before the system reboot) you just need to edit the file /etc/systemd/journald.conf and in the section [Journal] change the value of the Storage entry from auto to persistent.


[Journal]
Storage=persistent

That will do the trick!

You just need to restart the journald service in order to make the new setting effective:

systemctl restart systemd-journald

A GNOME Shell extension to tell whether GNOME is running under Xorg or Wayland

Since the release of Fedora GNU/Linux 25 in November 2016, Wayland has become the default choice for Gnome Shell sessions (where supported by the underlying hardware stack).

Altough GNOME Shell under Wayland works very well (actually, better than under Xorg), it is sometime required (or preferred) to run Xorg instead  (ie, some applications requires Xorg, for example TeamViewer).

So, I often wondered whether I was running Wayland or Xorg.

Since I’m lazy I developed a very simple GNOME Shell extension which does exactly that.

Here’s how it looks like:

That’s my first GNOME Shell extension (also, I don’t quite like JavaScript…) so it is probably far from perfect!  The source code is publically available on GitHub. Suggestions and pull requests are very welcome!

 

Analyze Tomcat logs with GoAccess

GoAccess is an open source real-time web log analyzer and interactive viewer that runs in a terminal in *nix systems.

It also can generate nice looking HTML reports.

I use it to analyze Tomcat servers logs and it works great.

If you’re running Fedora or RHEL/CentOS with EPEL repository enabled  you can easily install it with a simple:

 yum install -y goaccess

Next step is configure Tomcat to generate a suitable log file. I typically use this configuration in my server.xml


Then you can ask goAccess to read the log.

If you are interested in live monitoring just have to use this command (changing, of course, the date part of the file name and possibly the full path of your tomcat log directory):

goaccess -f /var/log/tomcat/localhost_access.2016-05-12.log \
--log-format='%h %^[%d:%t %^] "%r" %s %b "%R" "%u"' \
--date-format='%d/%b/%Y' \
--time-format='%H:%M:%S' 

If you want to generate a fancy HTML report, just add the -a option and redirect the output to a convenient file:

goaccess -f /var/log/tomcat/localhost_access.2016-05-12.log \
--log-format='%h %^[%d:%t %^] "%r" %s %b "%R" "%u"' \
--date-format='%d/%b/%Y' \
--time-format='%H:%M:%S' \
-a > report.html

that’s it!

Here’s an example of a such generated report.

I suggest you to take a look at the man page and/or at the documentation for more options and features.

La naskiĝo de nova vorto

Kiam ni parolas esperante, ni kutimas crei novajn vortojn. Tiele ne okazas kiam ni uzas nian gepatran lingvon.

Se vi havas italajn konatojn en via sociaj retoj, verŝajne lastatempe vi legis ion pri la vorto “petaloso”. Tiu vorto – kiu oni povas diri esperante “petalplena” – ne eksistas en la itala vortaro.

Ĉi tio estas bela rakonto pri la naskigo de nova vorto en la itala lingvo ke tute rapide “virusiĝi”.

Dum kelkaj semajnoj, instruistino Margherita Aurora publikigi ĉi tio post en Facebook:

Kelkaj semajnoj, laborante pri adjektivoj, unu el miaj lernantoj skribis pri floro kiu estas “petaloso“. La vorton, kvankam neekzistanta, plaĉis al mi, do mi sugestis lin sendi ĝin al la Accademia della Crusca por juĝado. Hodiaŭ ni ricevis respondon, preciza kaj  plencompleta. Dankon al mia malgranda inventisto Matteo.

Accademia della Crusca estas itala asocio kiu tasko estas (laŭ vikipedio), la “studado kaj rezervado de la itala lingvo

Jen kion ili renspondis:

Kara Matteo,

la vorto kiu vi inventis estas bone formita kaj povus esti uzita en la Itala lingvo tiel, kiel estas uzita samformitaj vortojn.

Vi kunigis vortojn petalo + oso -> petaloso = pieno di peli, con tanti petali

(esperante: petalo + plena -> petalplena = plena ol petaloj, kun multaj petaloj)

tielmaniere itala estas:

pelo + oso -> peloso = pieno di peli

(esperante: haro + plena -> harplena)

coraggio + oso -> coraggioso = pieno di coraggio, con tanto coraggio

Via vorton estas bela kaj klara, sed ĉu vi scias kiel vorto eniras en la vortaron? Nova vorto ne eniras en la vortaro kiam oni kreas ĝin, ankaŭ se gi estas “bela” kaj utila vorto. Por li eniri en la vortaro, fakte, necesas che la nova vorto ne estas konata kaj uzata nun el kiu kreis ĝin, sed ke multe da personoj uzas ĝis kaj multe da personoj ĝin komprenas. Se vi sukcesos diskonigi via vorton inter multa personoj kaj multaj personoj komencos diri kaj skribi “Com’è petaloso questo fiore!” au, tiel vi sugestas “le margherite sono fiori petalosi, mente i papaveri non sono molto petalosi”, jen, tiam do “petaloso” fariĝita itala vorto, ĉar italoj konas ĝin kaj ĝin uzas. Tiam, kiu redaktas vorarojn enigos la novan vorton kun la aliaj kaj disetendos lia signifo.

Tio estas kion ĝi funkcias: ne estas spertuloj, kiu fariĝas vortojn, kiu elektas kiujn vortojn estas belaj au malbelaj, utila kaj malutila. Kiam nova vorto estas en la buŝo el ĉiuj (aŭ el multaj), tiam la spertulo komprenas ke tio vorto iĝis vorto kial la aliaj kaj li enigas ĝin en la vortaro.

Mi esperas ke ĉi tio respondo utilas al vi, kaj mi sugestas al vi pli unu afero: bonan libron titolita “Drilla” kaj skribita el Andrew Clemens. Legu lin, eble kun via lerneja gekamarodoj kaj via instruistino: ĝi temas pri rakonto kial via, rakonto pri infano kiu kreis vorton kaj klopodas ĝin enigi en la vortaro.

Dankon pri skribi,

kara saluton al vi, al viaj lerneja gekamarodoj kaj al via instruistino.

Deduplicating Maven, Gradle and IDE’s JAR files on BTRFS

If you are a Java (or a Groovy) developer, your home directory will probably contains (at least) some GBs of downloaded JAR files.

Every dependency management tool (and IDE) downloads its own copy of the JAR files he need in a different directory. This lead to the situation where you might have several copies of the same file wasting your precious disk space!

In such situation one can take advantage of one of the nicest features of the BTRFS filesystem: deduplication.

Deduplication is not a feature supported by the official BTRFS’s tools yet, although the filesystem itself, being a COW filesystem, provides all the needed capabilities.

To leverage them, there’s an handy tool called duperemove.

Basically, what this amazing tool does is scan one or more directories (or subvolumes), find same content extents and deduplicate them. Deduplicate an extent mean that every reference to it points to the same phisical extent on disk (having hence only a phisical written content on disk).

To give you some numbers I’ll show how I used this on my commuting laptop (which has a small 300GB disk, so every GB is precious)

In my home have the following directories (related to the topic):

.eclipse
.gradle
.grails
.groovy
.IdeaIC13
.IdeaIC14
.IntelliJIdea13
.IntelliJIdea14
.ivy2
.m2

for a total usage of 4.8GB.

I used duperemove to deduplicate them with this command:

./duperemove -rdh /home/federico/{\
.eclipse,\
.gradle,\
.grails,\
.groovy,\
.IdeaIC13,\
.IdeaIC14,\
.IntelliJIdea13,\
.IntelliJIdea14,\
.ivy2,\
.m2\
}

after the execution the result was:

Comparison of extent info shows a net change in shared extents of: 704.0M

which is roughly a 14%, but, as they say, YMMV.

One might also suggest that I clean those directory from time to time, or just delete old releases, but  when I’m commuting I work offline and being able to patch and build some project which I’ve not been working on recently is quite handy. Also, having a small HD, even 700M might be useful!

UPDATE: including /usr/share/java(where my OS stores all packaged java libraries) and /usr/local (where I usually install binary distributions of software) further improves results.

Spring Integration – FTP files not reprocessed and broken flow

In the last years I’ve done everything I could to avoid using FTP as a mechanism to pass data from one procedure to the other inside our company.

Tools such Spring Integration and RabbitMQ helped me a lot in this process (making my life so much better)

However we still have to use FTP as exchange mechanism with several external partners.

Spring Integration has a wonderful FTP module that covers all of my needs, however I found a case which required me some googling to shed light on.

I had this scenario: a partner produces a file every n minutes and store in its own FTP server, with a never changing file name. I regularly check the FTP, get the file locally and put it in a RabbitMQ exchange.

So, basically, I have this kind of configuration (not the actual configuration, just an example):



        




     
        
             
        
    




this works nicely, with an important exception: if, for any reason, the outbound gateway cannot deliver the file to RabbitMQ (for example if the server is unreacheable, or restarting, or in maintenance) then this is what happens (suppose that the interesting file is called foo.csv):

  1. the file foo.csv is moved from remote server to the local filesystem
  2. the files is passed from the FTP inbound to the RabbitMQ outbound
  3. the FTP inbound marks the file as “processed”
  4. the RabbitMQ outbound fails to deliver the message to the broker (so it doesn’t delete the local file)

this seems the wanted behaviour (and, it is!), anyway, the problems arises at the next polling cycle:

  1. the FTP inbound sees that he already have a local file named foo.csv, so he won’t dowload the new one from the remote server
  2. anyway, it doesn’t either process the local foo.csv because he marked it as already “processed”

at this point, or chain is stuck and won’t start working again unless we restart our application.

The solution comes from the documentation (as always!)

From that page you can learn that the FTP inbound uses a FileListFilter to decide which local file process. Such filter can be changed setting the local-filter attribute. The default  filter is an AcceptOnceFileListFilter, which, as stated by its name, accept a file only once.

The list of accepted files is stored in memory, and that’s why restarting the application makes our foo.csv file processed again (of course, in case you need it, there are solutions to it persistent)

Going back to our problem, the solution is to set the filter to an instance of the class AcceptAllFileListFilter.





        


With this configuration in place, the remote foo.dat file won’t be downloaded unless the local copy has been deleted, and the local copy will be deleted after the first polling cycle which can succesfully deliver its content to the RabbitMQ broker.

An haptic notification device for Grails (but not only)

I am a commuter and during my commuting I often work on some Grails projects.

As Grails users knows, there are tasks that requires a rather big amount of time (re-run an app in non-interactive mode, run a test suite etc…) even on fairly recent hardware.

Instead of watching the screen during these interval of time I prefer to lay my eyes somewhere else for at least a couple of good reasons: the first one is that programmer’s eyes are always greedy for some rest, the second one is that I live (and commute through) one of the most beatiful place in the world and taking some relaxing view of the sight is amazing!

So I needed a way to know when my attention should go back to the screen  without the need to constantly check it.

The first obvious option was to produce some kind of audible notification (and such solution are already there for Grails), but this has two problems. The first one is that  volumes of music and notifications always mismatch so it happens that the notification sound is either too low or too high. The second one it that when I’m not using hearpones I don’t want to annoy other travellers with my notification sounds.

So I got the idea of buid my own “haptic notification device”.

Hardware

The first attempt with such device was a cheap bluetooth bracelet that, once paired with your smartphone, was ringing on every call.

The idea was to pair it with the PC and send a “RING” command whenever I needed to notify something.

Unfortunately I never managed it to work, and that’s a pity because that hardware was really small and light.

I haven’t really surrender with it, but meanwhile I decided to build my own vibrating box.

In the beginning I thought to use an Arduino Nano with a bluetooth module, but it required batteries and the size of the project wasn’t small enough to comfortably fit into my pocket.

Then I used a spare programmable USB I/O port that was lying around in my drawers.

The device is capable of a lot of things, however I’m just using one of the digital output. Probably you can find something cheaper with less features (and if you do, please leave a comment and let me know: I’m interested in other similar devices!)

To achieve the vibration I used a 3V vibrating motor found in these small “walking” bugs that I bought for a couple of euros some  years ago.

bugs

and here’s the schematic of the hardware:

haptic

 

I won’t explain the logic here (that is, anyway, pretty simple), but basically what happens is that setting the output port A2 to the “high” state, turns the motor on

And here’s how I packaged everything:

haptic_box

Software

Programming the USB I/O device is quite easy. I wrote a quick and dirty shell script (/usr/local/bin/hapticnotify.sh) that turns it rapidly on and off for a couple of times.

#!/bin/bash

# the serial name: might be different on your system
PORT=/dev/ttyACM0

# the number of cycles on/off
N_CYCLES=2

# the duration of a single vibration
DELAY=0.2

# configure the serial port parameters
stty -F $PORT cs8 57600 -ignbrk -brkint -icrnl -imaxbel -opost -onlcr -isig -icanon -iexten -echo -echoe -echok -echoctl -echoke noflsh -ixon -crtscts -hupcl

# send a reset command to the I/O device
echo "R" > $PORT

# configure every port to output mode
echo "C,0,0,0,0" > $PORT

# produce a sequence of short vibrations
for i in `seq 1 $N_CYCLES`; do

    # vibrate!
    echo "PO,A,2,1" > $PORT
    sleep $DELAY

    # mute!
    echo "PO,A,2,0" > $PORT
    sleep $DELAY
done

The script makes use of stty that is commonly packaged in GNU/Linux distribuition (at least, it is in Fedora 21, which I’m running)

Integration with Grails

Integrating with Grails has been straithgforward thanks to the power of the framework events management.

Events are triggered during executions of Grails target and allows you to hook custom event handlers.

Handlers can be defined inside the project or on a per-user basis.

I used the second approach so that all my project would use my haptic notification box without any modification.

The Grails events I was interested in were:

  • the project is running
  • a test suite has finished running
  • a war building is complete

so what I did was to create a file named $HOME/.grails/scripts/_Events.groovy with the following content:

String HAPTIC_SCRIPT="/usr/local/bin/hapticnotify.sh"

eventStatusFinal = { msg ->
    HAPTIC_SCRIPT.execute()
}

eventTestProduceReports = { msg ->
    HAPTIC_SCRIPT.execute()
}

and that’s all!

The only possibly required step is to make sure your user has the privileges to use the serial ports (in Fedora all you have to do is to add the user to the dialout group)

Jawbone UP24 with Samsung Galaxy Nexus

In case someone is wondering: yes, the jawbone UP 24 works perfectly with the Gnex, but only if you are running CyanogenMod on it.

The (old) Galaxy Nexus has a BLE (Bluetooth 4 Low Energy) device, but, as far as I know, the BLE feature is not enabled with stock Android, as it seems to be with CyanogenMod.

I bought my UP24 today and so far I had any issue at all to make it work with my Samsung Galaxy Nexus.

Dropbox hint: enabling LAN mode (in Linux)

If you are using Dropbox with multiple machines living in the same network, you can get advantage from the so-called LAN mode.

Basically, if you need to sync some Dropbox’s content between those machines, they’ll talk directly each one with the other instead of talking with the Dropbox servers.

This will drastically decrease the time to finish the operation (and reduce your Internet bandwidth usage)

A typical Fedora GNU/Linux installation won’t make use of the Lan mode feature because by default the needed network ports are close by the firewall.

All you need to do to make it work is open the port 17500 for both TCP and UDP protocols on all the machines involved.

To change your firewall configuration you can edit the file

/etc/sysconfig/system-config-firewall

and add the bold lines to your configuration

# Configuration file for system-config-firewall
--enabled
--port=17500:tcp
--port=17500:udp


and issue a

service iptables restart

as root.

If you feel more comfortable with a GUI, then you can find the firewall configuration tool under the Application->Other->Firewall menu.

You have to click on “Other ports” on the left menu, and then click on the “Add” button two times adding the TCP protocol the first one, and the UDP the second one.

Firewall GUI for Dropbox

You can make sure everything is working right clicking the Dropbox icon in the notification area during a sync: you should see “(LAN)” beside syncing and …a much faster synching!

Update: … and by the way, if you haven’t a Dropbox account yet, get one using my referral link http://db.tt/bEYuwB8 . In this way you’ll get 250MB bonus space (and I get the same amount too :D )