echo $RANDOM ## Tag: Linux ### Fedora 22 Scientific Alpha Just tested the Fedora 22 Scientific Alpha RC3 image today with the test scripts/programs. Some screen shots follow: IPython notebook IPython notebook/SymPy/matplotlib plotting Pandas A complete list of all the software included is in the guide. Contribute to Fedora Scientific • Use it! • You can help complete the guide. One notable piece of software missing from that list is “pandas”. • You can add examples/scripts/IPython notebooks to the repository here ### LCA 2015 talk: Beaker’s Hardware Inventory system The video is up on YouTube: http://t.co/WorOwbv37w Since I could not make it to LCA, Nick Coghlan presented the talk on my behalf. Thanks Nick! ### A docker based workflow for working on beaker While working with beaker‘s code base, I often feel the need to run my tests for a patch/feature and continue to work on with different things while they run, including running other tests testing something different. Currently this is not possible since we start off with a clean database on every test run and simultaneous runs would obviously make one run step on another’s feet. I finally have an initial docker based prototype for making this possible. ### Book Review: Linux System Programming I received a review copy of the book as part of the blogger review program. Title: “Linux System Programming (2nd Edition) ” by Robert Love; O’Reilly Media. Summary: This book consists of 11 chapters. The first chapter introduces you nicely to the the core topics and lays the foundation for the rest of the book. Files (including some hints on the role of the virtual file system and how they are represented in the Kernel), Input/Output (User buffered I/O, I/O scheduling, Scatter-Gather I/O), Processes (including their creation mechanisms and management), Threads (and how Linux implements them along with a treatment of the POSIX threads library), Memory (Process address space, dynamic memory allocation strategies, and how they work, memory locking) form the core of the book. The second last chapter discusses signal handling. The last chapter of the book is on time (the different types of time, how you can get/set time, measure time elapsed and timers) and is sort of a “standalone” topic for the book. The first appendix discusses the GCC extensions to the C language and can be handy when you read the Kernel source code. Reactions: In this book, the author discusses some of the most important topics that one would want to learn about when venturing into the area of “system programming” on Linux. He introduces the topics in a friendly manner adding some fun anecdotes from time to time (what does the “c” in calloc() stand for?).At various places, the reader is given a peek under the hood (for example, pause() is one of the simplest system calls implemented) which can only make the curious reader happy and itchy to download the kernel source code and start grepping. The book includes code examples throughout and hence if you are learning a topic for the first time, these are very useful starting points. Verdict: System programming on Linux is an area encompassing number of related topics most of which can fill up whole books on their own. I also could not help comparing this book with “The Linux Programming Interface” by Michael Kerrisk (a book which I own already). Should you buy this book if you already own the latter? Yes, you should. While not being “encyclopedic” and not covering topics such as socket programming at all, Robert Love’s “Linux System Programming” has the right level of treatment and detail for the reader interested in system programming. Product page: ### poweroff, halt, reboot and systemctl On Fedora (and perhaps other Linux distros using systemd) you will see that the poweroff, reboot and halt commands are all symlinks to systemctl: > ls -l /sbin/poweroff /sbin/halt /sbin/reboot lrwxrwxrwx. 1 root root 16 Oct 1 11:04 /sbin/halt -> ../bin/systemctl lrwxrwxrwx. 1 root root 16 Oct 1 11:04 /sbin/poweroff -> ../bin/systemctl lrwxrwxrwx. 1 root root 16 Oct 1 11:04 /sbin/reboot -> ../bin/systemctl  So, how does it all work? The answer lies in this code block from systemctl.c: .. 5556 if (program_invocation_short_name) { 5557 5558 if (strstr(program_invocation_short_name, "halt")) { 5559 arg_action = ACTION_HALT; 5560 return halt_parse_argv(argc, argv); 5561 } else if (strstr(program_invocation_short_name, "poweroff")) { 5562 arg_action = ACTION_POWEROFF; 5563 return halt_parse_argv(argc, argv); 5564 } else if (strstr(program_invocation_short_name, "reboot")) { 5565 if (kexec_loaded()) ..  program_invocation_short_name program_invocation_short_name is a variable (GNU extension) which contains the name used to invoke a program. The short indicates that if you call your program as /bin/myprogram, it is set to ‘myprogram’. There is also a program_invocation_name variable consisting of the entire path. Here is a demo:  /*myprogram.c*/ # include <stdio.h> extern char *program_invocation_short_name; extern char *program_invocation_name; int main(int argc, char **argv) { printf("%s \n", program_invocation_short_name); printf("%s \n", program_invocation_name); return 0; }  Assume that the executable for the above program is created as myprogram, execute the program from a directory which is one level up from where it resides. For example, in my case, myprogram is in $HOME/work and I am executing it from $HOME: > ./work/myprogram myprogram ./work/myprogram  You can see the difference between the values of the two variables. Note that any command line arguments passed are not included in any of the variables. Back to systemctl Okay, so now we know that when we execute the poweroff command (for example), program_invocation_short_name is set to poweroff and this check matches: if (strstr(program_invocation_short_name, "poweroff")) ..  and then the actual action of powering down the system takes place. Also note that how the halt_parse_argv function is called with the parameters argc and argv so that when you invoke the poweroff command with a switch such as --help, it is passed appropriately to halt_parse_argv: 5194 static const struct option options[] = { 5195 { "help", no_argument, NULL, ARG_HELP }, .. .. 5218 case ARG_HELP: 5219 return halt_help();  Fooling around Considering that systemctl uses strstr to match the command it was invoked as, it allows for some fooling around. Create a symlink mypoweroff to /bin/systemctl and then execute it as follows: > ln -s /bin/systemctl mypoweroff > ./mypoweroff --help mypoweroff [OPTIONS...] Power off the system. --help Show this help --halt Halt the machine -p --poweroff Switch off the machine --reboot Reboot the machine -f --force Force immediate halt/power-off/reboot -w --wtmp-only Don't halt/power-off/reboot, just write wtmp record -d --no-wtmp Don't write wtmp record --no-wall Don't send wall message before halt/power-off/reboot  This symlink is for all purpose going to act like the poweroff command since systemctl basically checks whether ‘poweroff’ is a substring of the invoked command. To learn more, see systemctl.c Related Few months back, I had demoed invoking a similar behaviour in programs where a program behaves differently based on how you invoke it using argv[0] here. I didn’t know of the GNU extensions back then. ### Managing IPython notebook server via systemd: Part-I If you are using IPython notebook on a Linux distribution which uses systemd as it’s process manager (such as Fedora Linux, Arch Linux) , you may find this post useful. I will describe a fairly basic configuration to manage (start/stop/restart) IPython notebook server using systemd. Creating the Systemd unit file First, we will create the systemd unit file. As root user, create a new file  /usr/lib/systemd/system/ipython-notebook.service and copy the following contents into it:  [Unit] Description=IPython notebook  [Service] Type=simple PIDFile=/var/run/ipython-notebook.pid ExecStart=/usr/bin/ipython notebook --no-browser --pylab=inline User=ipynb Group=ipynb WorkingDirectory=/home/ipynb/notebooks [Install] WantedBy=multi-user.target  Note that due to the naming of our unit file, the service will run as ipython-notebook. To completely understand the above unit file, you will need to read up a little of the topic. You may find my earlier post useful which also has links to systemd resources. Three things deserve explanation though: The line,  ExecStart=/usr/bin/ipython notebook --no-browser --pylab=inline specifies the command to start the IPython notebook server. This should be familiar to someone who uses it. The lines, User=ipynb and Group=ipynb specify that we are going to run this process as user/group ipynb (we create them in the next step). The line WorkingDirectory=/home/ipynb/notebooks specify that the notebooks will be stored/server in/from /home/ipynb/notebooks Setting up the user As root, create the user ipynb:  # useradd ipynb  Next, as ipynb, create a sub-directory, notebooks:  # su - ipynb [ipynb@localhost ~]$ mkdir notebooks [ipynb@localhost ~]\$ exit 

Starting IPython notebook

We are all set now to start IPython notebook. As the root user, reload all the systemd unit files, enable the ipython-notebook service so that it starts on boot, and then start the service:

 # systemctl daemon-reload # systemctl enable ipython-notebook ln -s '/usr/lib/systemd/system/ipython-notebook.service' '/etc/systemd/system/multi-user.target.wants/ipython-notebook.service' # systemctl start ipython-notebook 

If you check the status of the service, it should show the following:

 # systemctl status ipython-notebook ipython-notebook.service - IPython notebook Loaded: loaded (/usr/lib/systemd/system/ipython-notebook.service; enabled) Active: active (running) since Sun 2013-09-22 22:39:59 EST; 23min ago Main PID: 3671 (ipython) CGroup: name=systemd:/system/ipython-notebook.service ├─3671 /usr/bin/python /usr/bin/ipython notebook --no-browser --pylab=inline └─3695 /usr/bin/python -c from IPython.zmq.ipkernel import main; main() -f /home/ipynb/.ipython/profile_default/security/kernel-6dd8b338-e779-4e67-bf25-1cd238... Sep 22 22:39:59 localhost ipython[3671]: [NotebookApp] Serving notebooks from /home/ipynb/notebooks Sep 22 22:39:59 localhost ipython[3671]: [NotebookApp] The IPython Notebook is running at: http://127.0.0.1:8888/ Sep 22 22:39:59 localhost ipython[3671]: [NotebookApp] Use Control-C to stop this server and shut down all kernels. Sep 22 22:40:21 localhost ipython[3671]: [NotebookApp] Using MathJax from CDN: http://cdn.mathjax.org/mathjax/latest/MathJax.js Sep 22 22:40:22 localhost ipython[3671]: [NotebookApp] Kernel started: 6dd8b338-e779-4e67-bf25-1cd23884cf5a Sep 22 22:40:22 localhost ipython[3671]: [NotebookApp] Connecting to: tcp://127.0.0.1:51666 Sep 22 22:40:22 localhost ipython[3671]: [NotebookApp] Connecting to: tcp://127.0.0.1:52244 Sep 22 22:40:22 localhost ipython[3671]: [NotebookApp] Connecting to: tcp://127.0.0.1:44667 Sep 22 22:40:22 localhost ipython[3671]: [IPKernelApp] To connect another client to this kernel, use: Sep 22 22:40:22 localhost ipython[3671]: [IPKernelApp] --existing kernel-6dd8b338-e779-4e67-bf25-1cd23884cf5a.json 

You should now be able to access IPython notebook as you would normally do. Finally, you can stop the server as follows:

 # systemctl stop ipython-notebook 

The logs are redirected to /var/log/messages:

 Sep 22 22:39:59 localhost ipython[3671]: [NotebookApp] Created profile dir: u'/home/ipynb/.ipython/profile_default' Sep 22 22:39:59 localhost ipython[3671]: [NotebookApp] Serving notebooks from /home/ipynb/notebooks Sep 22 22:39:59 localhost ipython[3671]: [NotebookApp] The IPython Notebook is running at: http://127.0.0.1:8888/ Sep 22 22:39:59 localhost ipython[3671]: [NotebookApp] Use Control-C to stop this server and shut down all kernels. Sep 22 22:40:21 localhost ipython[3671]: [NotebookApp] Using MathJax from CDN: http://cdn.mathjax.org/mathjax/latest/MathJax.js Sep 22 22:40:22 localhost ipython[3671]: [NotebookApp] Kernel started: 6dd8b338-e779-4e67-bf25-1cd23884cf5a Sep 22 22:40:22 localhost ipython[3671]: [NotebookApp] Connecting to: tcp://127.0.0.1:51666 Sep 22 22:40:22 localhost ipython[3671]: [NotebookApp] Connecting to: tcp://127.0.0.1:52244 Sep 22 22:40:22 localhost ipython[3671]: [NotebookApp] Connecting to: tcp://127.0.0.1:44667 Sep 22 23:05:35 localhost ipython[3671]: [NotebookApp] received signal 15, stopping Sep 22 23:05:35 localhost ipython[3671]: [NotebookApp] Shutting down kernels Sep 22 23:05:35 localhost ipython[3671]: [NotebookApp] Kernel shutdown: 6dd8b338-e779-4e67-bf25-1cd23884cf5a

 

For me, the biggest reason to do this is that I do not have to start the IPython notebook server everytime on system startup manually, since I know it will be running when I need to use it. I plan to explore managing custom profiles next and also think more about a few other things.

### Notes on writing systemd unit files for Beaker’s daemon processes

Recently, I had a chance to write systemd unit files for the daemon processes that run as part of Beaker: beakerd which is the scheduling daemon running on the server and the four daemons running on the lab controller – beaker-proxy, beaker-provision, beaker-watchdog and beaker-transfer.

This post may be of interest to you if you are using python-daemon to write programs which are capable of running as daemon processes and you want to write systemd unit files for them.

beakerd’s unit file

Here is the systemd unit file for beakerd, which I will use to illustrate the core points of this post. The other unit files are similar, and hence I will explain only where they differ from this one:

[Unit]
Description=Beaker scheduler
After=mysqld.service

[Service]
Type=forking
PIDFile=/var/run/beaker/beakerd.pid
ExecStart=/usr/bin/beakerd
User=apache
Group=apache

[Install]
WantedBy=multi-user.target

The [Unit] section has a description of the service (using the Description option) and specifies that it should start after the mysqld.service has started using the After option. beakerd needs to communicate to a MySQL server before it can start successfully. It can work with a local or a remote MySQL server. Hence, specifying After sets up an ordering that if there is a local MySQL server, then wait for it to start before starting beakerd. Using Requires is not suitable here to accommodate the possibility that beakerd may be configured to use a remote MySQL server.

In the [Service] section, the Type is set to Forking. This is because, beakerd uses python-daemon which forks itself (detaches itself) during the daemonization. However, you must ensure that when creating a DaemonContext() object, you should specify detach_process=True. This is because, if python-daemon detects that it is running under a init manager, it doesn’t detach itself unless the keyword is explicitly set to True, as above (you can see the code in daemon.py). Hence, although not setting the above keyword would work under SysV Init, it doesn’t work under systemd (with Type=Forking), since the daemon doesn’t fork at all and systemd expects it to fork (and finally kills it). The PIDFile specifies where the process ID is dumped by beakerd and is setup while creating the DaemonContext object as follows and ExecStart specifies the location to the binary that is to be started.

The beakerd process is to be run as the apache user and group, which is specified by the User and Group options.

In the [Install] section, the WantedBy option specifies when the beakerd process should be started (similar to the concept of “run levels” in SysV init). systemd defines several targets, and here we define that we want beakerd to start as part of the multi user setup.

That’s all about beakerd’s unit file.

beaker-provision’s unit file

beaker-provision and the other daemons running on the lab controller have similar unit files:

[Unit]
Description=Beaker provisioning daemon
After=httpd.service

[Service]
Type=forking
PIDFile=/var/run/beaker-lab-controller/beaker-provision.pid
ExecStart=/usr/bin/beaker-provision
User=root
Group=root

[Install]
WantedBy=multi-user.target

All the four lab controller daemons need to communicate with Beaker web application – which can be local or remote, and hence the After option specifies the dependency on httpd.service. And, this particular daemon runs as root user/group, which is specified by the User and group options.

And everything else is similar to beakerd’s unit file and also the other lab controller daemons.

Shipping SysV init files and systemd unit files in the same package

The beaker packages ship both SysV init files and systemd unit files now so that it can use systemd when available, but use SysV init otherwise. This commit can give you some idea of how to go about it.

systemd resources