Development environment for BSB

Table of Contents

• tl;dr:
• Introduction
• Using the lab computers remotely
  • SSH
    • Using SSH keys
    • SSH-Config
  • Shared tmux
  • SSHFS
• Required software for your own installation
• Build OO/MPStuBS handout
• Testing and debugging OOStuBS (Make-Targets)
• Source code management in a common Git for the whole group

tl;dr:

• SSH to our lab is available

    $ ssh -J and.tanenbaum@lab.sra.uni-hannover.de and.tanenbaum@lab-pc04 # for Lab-PC 4
  

• shared tmux

    $ tmux -S /tmp/my-shared-tmux new                       # create tmux session
    $ setfacl -m "u:and.tanenbaum:rw" /tmp/my-shared-tmux   # give access to Andrew Tanenbaum
  
    $ tmux -S /tmp/my-shared-tmux attach                    # connect to the session (execute from other computer)
  

• For your own PC:

    $ apt install nasm qemu qemu-system-x86 qemu-kvm build-essential binutils gcc-multilib g++ g++-multilib
  

  • QEMU (qemu, qemu-system-x86, qemu-kvm)
  • GCC (build-essentials, binutils, gcc-multilibs, g++)
  • nasm

Introduction

All the tools required for working on the lab exercises (and therefore for the development of OOStuBS and MPStuBS) are pre-installed in the lab. The lab computers are accessible via SSH. The setup details can be found below. You can of course also work on the tasks at home, we recommend using Linux for this. Below you will find some tips on how to configure your Linux computer accordingly.

Attention

If you want to install the software at home, you are of course fully responsible for any problems. Failed installations will not be accepted by us as an excuse for late program delivery.

As errors can sometimes creep in during operating system development, you must test your solutions before submitting them. We use an emulator (QEMU or KVM) for this purpose. We always use the QEMU and KVM on the lab to check your solution. You should therefore always test at least once in the lab, this is the reference platform!

Using the lab computers remotely

The lab computers can be accessed via SSH.

SSH

To use SSH, you need a computer number. Take your group number modulo 10 (to achieve a bit of load balancing). Then use the following ssh command (with a jump host)

$ ssh -J <euer_login_name>@lab.sra.uni-hannover.de <euer_login_name>@lab-pc<computer number>

I.e., for Andrew Tanenbaum the login would be for computer 4:

$ ssh -J and.tanenbaum@lab.sra.uni-hannover.de and.tanenbaum@lab-pc04

Using SSH keys

Gitlab prohibits anonymous access to your repository. Even a normal login, as is sometimes the case with GitHub, is not possible on the repositories. You must therefore create a key pair and provide Gitlab with the public key for authentication.

The best way to do this is to generate a key locally using ssh-keygen:

$ ssh-keygen -t ed25519

Either the key is stored as ~/.ssh/id_ed25519 in your home in the hidden directory .ssh, or you give it a different name. id_ed25519 is used by ssh by default, all other names do not exist for ssh at first. If you enter a different name, you must make it known to the ssh-agent, for example, by executing the following:

$ eval $(ssh-agent) # start ssh-agent
$ ssh-add ~/.ssh/gitlab # make the key for Gitlab known

In the second step, you may be asked for the password that you entered when generating the key.

Once you have added the public key to your Gitlab preferences, you can simply execute all git commands. From then on, authentication is done implicitly via the keys, and you no longer have to enter passwords.

SSH-Config

We recommend adding the lab computer to the local SSH config. To do this, the following entry must be added to the file ~/.ssh/config:

host lab-pc04
  User and.tanenbaum
  ForwardX11 yes
  ProxyJump and.tanenbaum@lab.sra.uni-hannover.de

Please adjust the host name for the specific computer.

Once the key and SSH config have been set up, you can simply connect to the SRA computer via ssh lab-pc04.

Shared tmux

For collaborative work, you can use tmux to share a TTY session with another user. Both users can then see and edit the same terminal. In addition, tmux works session-based, i.e. you share not just one terminal, but an entire session, specifically any number of terminals and their status and layout.

To set up, create a tmux session by specifying a socket (-S / -L):

$ tmux -S /tmp/my-shared-tmux new

This creates a tmux server if no one is listening on the socket yet. (The socket name my-shared-tmux should be replaced by your own name)

With the option -s session-name (after new) the session can also be assigned its own name (default: "0").

Additional users or tmux instances can now connect to the session:

$ tmux -S /tmp/my-shared-tmux attach

If necessary, the session must be identified with the -t session-name option. A detach is done with the key sequence Ctrl-b d.

For another user to be able to attach, they must have the appropriate rights to the socket, which can be achieved using ACLs:

$ setfacl -m "u:and.tanenbaum:rw" /tmp/my-shared-tmux

(where other.user is replaced with the user name of the partner).

Please be careful when granting permissions and do not use UNIX group permissions in the lab. Remember that everyone in such a group will then have full access to your tmux session.

For more information see the official tmux documentation or the tmux man page.

SSHFS

If you want to use a graphical editor (locally) but still want to use the make targets on the lab computer, it is a good idea to mount the lab locally using SSHFS. This allows you to edit your code locally as usual and still execute the make targets remotely.

In principle, it is sufficient to execute something like this:

$ mkdir sra-mount
$ sshfs lab-pc01: sra-mount
$ cd sra-mount; ls

Required software for your own installation

As specified in the Makefile, g++ or clang can be used for compiling, and the Netwide Assembler (nasm) for assembling the startup code and the hardware-related subprograms. The x86 emulator QEMU is suitable for testing and, thanks to a built-in GDB stub, also for debugging with the GNU Debugger. The appropriate environment is available in the lab; if you want to do the whole thing at home, you have to set up the software mentioned accordingly. If you have any problems, please feel free to ask us.

Build OO/MPStuBS handout

• From task 1 onwards, there is one handout for OOStuBS and one for MPStuBS. The handouts are Git repositories on the Uni Gitlab, into which we successively check in the initial commit for each task.

  We will create repositories for your groups in Gitlab and you can ask the instructor to assign you to these groups. The description of the group repository shows how you can integrate the handout (either OOStuBS or MPStuBS) into your repository.

• For each task, we will extend the handout repositories so that you only have to pull the changes for the respective task into your repository.
• All handouts that you receive from us can be compiled correctly, but only contain incomplete code. You must therefore complete the code in the correspondingly marked functions and classes for each task.
• The actual compilation of OOStuBS is done by calling make in the source directory. All .cc and .asm files in the solution directory are then compiled with the appropriate tools (compiler or assembler) and linked together as a bootable system image. The Makefile also provides commands to test or debug the system using QEMU. The exact commands are provided by make help.

Testing and debugging OOStuBS (Make-Targets)

• The quickest and easiest way to test your implementation is to use make kvm to run your system image in QEMU with hardware virtualization:

    $ make kvm
  

  By default, QEMU is configured to emulate a system with four processors. For the development of OOStuBS, this is not a problem, as the additional CPUs are simply "left alone" without any further action. For MPStuBS, the KVM mode allows your system to run in parallel on multiple cores. This test therefore comes relatively close to the test on the real hardware in terms of race conditions and faulty synchronization.

Attention

To quit a QEMU that was started in curses mode, you have to switch to the monitor view (Alt-2) and enter quit there. To return from the monitor view to the CGA view, press Alt-1. The serial interface is displayed in another virtual QEMU window (Alt-3).

• For easier troubleshooting, hardware virtualization can also be disabled by using the make qemu command instead. In this mode, the guest system is only emulated pseudo-parallel, which simplifies the hunting for more serious bugs, but can also mask existing bugs that would otherwise only occur with make kvm or on real hardware.

• If you get stuck with simple "`printf` debugging", you can use the GDB stub integrated in QEMU to connect to the emulation with a debugger (gdb). This way you can easily run your operating system code step by step to find out the reason for any crashes or unwanted behavior. We provide the target gdb in the Makefile for this purpose:

  $ make gdb
  

  In this configuration, the GDB stub in the emulator waits for a socket connection via which a gdb can connect to the emulator in order to debug the system. The debugger is already started by the Makefile, so that the gdb prompt appears in the terminal immediately after starting QEMU.

  A short reference of the GDB functions can be found here. If you want detailed instructions on how to use a particular GDB command, you can use the help function built into GDB:

  (gdb) help <command name>
  

  Note: As the operating system is paused by the QEMU emulator when using the GDB stub, you must not restart the program execution in GDB with run, but must continue it with continue instead.

  For a quick overview of the register and stack contents, it is also advisable to store this gdbinit file under the name .gdbinit in your own home directory. Various view options at the beginning of the file can be changed to suit your own taste. Another useful alternative is the GDB dashboard.

  GDB can also send commands to QEMU using monitor. Practical here is e.g: monitor info registers (show registers), monitor system_reset restart system without restarting GDB.

• The easiest way to work from home is via an SSH connection. But within the SSH session, you cannot start a window on this connection by yourself. To get around this limitation, QEMU is able to map the CGA screen in text mode completely to a terminal with NCurses. The Makefile targets are therefore set up so that they automatically switch to NCurses mode if no display is available.

  However, the GDB target requires an X11 window. Therefore, it is possible to start GDB separately and instruct QEMU to wait for GDB. To start gdb standalone, the existing targets can be used with the -gdb suffix. To connect, the special target connect-gdb is available. This means that to start and debug a QEMU with your system, you open two terminals via SSH and enter one of them:

  $ make qemu-curses-gdb
  

  and in the other:

  $ make connect-gdb
  

• Note on compiler optimizations: The default behavior of the default build system is to use the -O3 optimization level. In the case of persistent bugs, the resulting machine code can make debugging considerably more difficult. For these cases, all the Makefile targets mentioned above are also available as variants with the -noopt suffix, which deactivates the compiler optimizations. GDB debugging without optimizations is possible with this call, for example:

  $ make gdb-noopt
  

• An overview of the available commands (in addition to reading the Makefile) is also provided by

    $ make help
  

Source code management in a common Git for the whole group

As already mentioned, the template for OOStuBS/MPStuBS/StuBSmI can be found in Uni-Gitlab. We have created a repository there for each exercise group, which you should use to work together.

To add the template repo to your repo, use the following commands:

$ git remote add handout git@gitlab.uni-hannover.de:sra/v_bsb_2024/oostubs.git # handout for single-core
# or
$ git remote add handout git@gitlab.uni-hannover.de:sra/v_bsb_2024/mpstubs.git # handout for multi-core

$ git pull template

You can find a brief overview of the Git commands here. For a deeper introduction to distributed source code management, we recommend the Pro Git book, which is available as Creative Commons.