Automatic translation-chek update crontab

I have a cron job that is responsible for updating the DokuWiki PHP translation checks page on dokuwiki.

Each day, this cron job updates a DokuWiki git repository clone, checks the DokuWiki site to guess if it is safe to send a page there, runs the checklanguage.php tests, and sends the page.

This page documents how I do this.

I always keep my cronjobs and their needed files in a separate directory. In this case, this is the MyCrontabDir directory.

In this directory, there are :

• dw-update-tr : the script run by the crontab
• helper/checklanguage.php : the php script that checks the translations
• helper/saptdvhf : a script that can send files to DokuWiki site,
• helper/dokuwiki : the DokuWiki git repository clone
• helper/dw-creds : a file that contains user:password to log in DokuWiki site
• helper/tmp : a temp dir.

The cronjob must be able to write to :

• helper/dokuwiki
• helper/tmp
• /tmp/log (eventually when checklanguage.php debug is activated)

MyCrontabDir
├── doc/
│   ├── documentation.src
│   ├── documentation.txt
│   └── Makefile
├── dw-update-tr*
└── helper/
    ├── checklanguage.php
    ├── dokuwiki/
    │   ├── bin/
    │   ├── checklanguage.php
    │   ├── conf/
    │   ├── COPYING
    │   ├── _cs/
    │   ├── data/
    │   ├── doku.php
    │   ├── feed.php
    │   ├── .git/
    │   ├── .gitignore
    │   ├── .htaccess.dist
    │   ├── inc/
    │   ├── index.php
    │   ├── install.php
    │   ├── lib/
    │   ├── README
    │   └── _test/
    ├── dw-creds
    ├── saptdvhf*
    └── tmp/
        └── resultat

12 directories, 17 files

• bash
• test, mkdir, cp, rm, ….
• curl
• sed
• awk
• md5sum
• git
• php

At DokuWiki, it is possible to prevent my own crontab to work ! Yes remote control is available…

The cron job uses a dedicated dokuwiki account to send the page. Disabling this account will prevent the cron job to work. current account is schplrtz.au

Another solution is to modify, the script on the dokuwiki page. The cronjob tests if there was a modification before sending its result page. For example, on DokuWiki page, adding a line just below “<?php” in the script that reads

// no more automatic update please

will prevent my own cronjob to run. If this line is removed and the script comes back to the exact version it was, then my cronjob will run again.

Isn't that beautiful ?

The script that glues all this together is not very complicated. There are 2 things to note about it, tough.

It sets TZ variable in a form that is suitable to both the system it is running on, and “php -d 'date.timezone'” ini parametre.

Before actually running the php test and sending the result page to dokuwiki, it tries to download the checklanguage.php script from dokuwiki. If the downloaded (or not downloaded) script md5sum is different from a known md5sum, then obviously, the script on the page has been modified (or removed), and the script should not run nor send itself to DokuWiki site.

NB
Even when successful, the cron job will print things to stderr, mainly because helper/saptdvhf does so.

dw-update-tr

#! /bin/bash
set -e # stop at first error
 
IFS=' 	
'
PATH=/usr/bin:/bin
myself=${0}
case "$myself" in
(/*) : ;;
(*) myself=$PWD/$myself ;;
esac
 
here=${myself%/*}
mkdir -p "$here/helper/tmp"
# export ALL_PROXY=yourproxy.example.com:12345
#export TZ='Europe/Paris'
export TZ='UTC'
 
pageid=playground:unessai
pageid=teams:i18n:translation-check
 
cd "$here/helper/dokuwiki"
rm -f checklanguage.php
git pull
 
# Now, check that the page still exists and that code has not changed
# 1st grep the url of the codeblock that contains checklanguage.php
 
set -- $(
   {
      curl -s https://www.dokuwiki.org/$pageid || :
   } | sed -ne '/href.*export.*checklanguage.php/s/.*href="\([^"]*\)".*/\1/p'
)
# then download the code and compute its md5sum
set -- $(
   {
      curl -s \
         "https://www.dokuwiki.org$1" \
         2>/dev/null ||
      :
   } | {
      md5sum ||
      :
   }
)
# check md5sum of the script is acceptable
case "$1" in
(8774f57c1db518f00c31f37ddc1ec002 | ab6b15cba28a30c0bb827d3912609b7f | b9ec3070fd0e00c4c9adf6b873538699 | 8abf956295de1363af1d26e8b6deae39 | d27024fc625536d934092d4645f07376 )
   :
;;
(*)
   exit 1
;;
esac
 
# OK ! let's go marco.
ln ../checklanguage.php
php -d "date.timezone=$TZ" checklanguage.php > "$here/helper/tmp/resultat"
 
"$here/helper/saptdvhf" \
	-p "$here/helper/tmp/resultat" \
	-i "$here/helper/dw-creds" \
	https://www.dokuwiki.org/"$pageid"

50 6 * * * /bin/true /home/user/MyCrontabDir/dw-update-tr >/tmp/crlog 2>&1

Here is the archive that contains everything : update-dokuwiki-translation-check.tar.bz2

Untar it whereever it pleases you, all paths are relative.

Once it is untarred, there remains 2 things to do

1. get a dokuwiki git repository clone in the helper directory. See devel:git for more infos on that matter.

   cd update-dokuwiki-translation-check/helper
   git clone git://github.com/splitbrain/dokuwiki.git

2. create a file that contains dokuwiki login and password suitable to send pages to DokuWiki site. You should really create a dedicated account on dokuwiki.

   cd update-dokuwiki-translation-check/helper
   ./saptdvhf create-ident-file dw-creds