.. _installation:

Installation
============

Installation via Docker
-----------------------

Atomiq provides docker images that bundle everything needed to run atomiq inside the artiq ecosystem. This is probably
the easiest way to get atomiq and artiq running in a few simple steps:

#.  If not already done, install docker on your system and make sure that the docker daemon is running and your user
    has sufficient permissions to create and run docker containers.

    .. tip::

      :ref:`Here <install-docker-compose>` you can find a `Docker Compose <https://docs.docker.com/compose/>`_
      file to get you started even faster. The Compose file assumes a standard ARTIQ folder structure. To understand its
      structure, continue reading here.
      This Compose file should also work on Docker Desktop for Windows. But note that we do
      not officially support atomiq on windows.

#.  Assume you have your atomiq/ARTIQ experiment scripts and the `device_db.py` in the current folder, you can start the artiq_master by
    running:

    .. code-block:: bash

      $ docker run --rm -it \
                   -v `pwd`:/experiment \
                   -p 1380:1380 -p 1381:1381 -p 1382:1382 -p 3250:3250 -p 3251:3251 -p 1067:1067 \
                   registry.gitlab.com/atomiq-project/atomiq:stable \
                   python -m atomiq.frontend.atomiq_master --bind \* -r /experiment --device-db /experiment/device_db.py -v --heros-connect tcp/<IP_OR_HOSTNAME>:7447 --heros-realm <realm>

    .. note::

      The ``--heros-connect`` option allows you specify a zenoh peer that can be used to connect to the network of HEROs.
      If the container is run in the docker host network, UDP discovery is possible and this option is not necessary.

      .. tip::

        We recommend starting up a `zenoh router (zenohd) <https://zenoh.io/docs/getting-started/installation/>`_ as a peer.

      .. important::

        If using Docker on Windows (unsupported) it is always necessary to connect to a zenoh peer manually as WSL does not reliably support multicast (see for example this
        `Stack Overflow discussion <https://stackoverflow.com/questions/62108116/reroute-mdns-query-from-wsl-subnet-to-windows-host-subnet/62246694#62246694>`_).

      .. tip::

        Under Linux, it is also possible to run the atomiq master in a ``macvlan`` network. This requires more effort to setup, but makes multicast work without
        requiring the container running in the host network or specifying ``--heros-connect``. See :ref:`install-docker-compose` for more information.

    .. note::

       The optional ``--heros-realm`` (defaults to ``heros``) is used to set the HEROS `realm <https://heros-761c0f.gitlab.io/paradigms.html#realms>`_ of all :ref:`Atomiq related HEROs <atomiq-as-hero>`.


    Additionally, you might want to run the ``artiq_ctlmgr``. This is easily done with the following command:

    .. code-block:: bash

      $ docker run --rm -it \
                   -p 1066:1066 -p 3249:3249 -p 1383:1383 -p 1384:1384 -p 1385:1385 -p 1386:1386 \
                   registry.gitlab.com/atomiq-project/atomiq:stable \
                   artiq_ctlmgr --bind \* -s 172.17.0.1 -v

    Make sure that you set the host correctly for the ``core_moninj``, ``core_log``, and ``core_analyzer`` in ``your device_db.py``.

#.  Finally, if you want to use the artiq dashboard (with modifications from atomiq), you can also use the docker image ``atomiq:stable-full``.
    However, passing the GUI from the container to the host system needs a bit more complexity and depends on your
    precise setup.

    .. tip::

      You can also run the dashboard from a local python environment, without using docker, as described :ref:`here <install-dashboard-only>`


    Assuming you are running on linux with an X server, the following will start the dashboard

    .. code-block:: bash

      $ xhost +
      $ docker run -it --rm \
                   -e DISPLAY \
                   -v $HOME/.Xauthority:/qao/.Xauthority:rw \
                   -v /tmp/.X11-unix/:/tmp/.X11-unix \
                   registry.gitlab.com/atomiq-project/atomiq:stable-full \
                   python -m atomiq.frontend.atomiq_dashboard -s <ARTIQ_MASTER_IP_OR_HOSTNAME>

    Note that the xhost command comes with severe security implications as it essentially gives full access to your X
    server. Only do this in a trusted environment.

    If you are running wayland, the following command will launch the dashboard:

    .. code-block:: bash

      $ docker run -it --rm \
                   -e XDG_RUNTIME_DIR=/tmp \
                   -v $XDG_RUNTIME_DIR/$WAYLAND_DISPLAY:/tmp/$WAYLAND_DISPLAY \
                   -e "WAYLAND_DISPLAY" \
                   -e "QT_QPA_PLATFORM=wayland" \
                   registry.gitlab.com/atomiq-project/atomiq:stable-full \
                   python -m atomiq.frontend.atomiq_dashboard -s <ARTIQ_MASTER_IP_OR_HOSTNAME>


    Here, you will have to replace `<ARTIQ_MASTER_IP_OR_HOSTNAME>` with the IP or the hostname of the machine where your
    artiq_master is running.

In the above examples, the `stable` tag is used. This will run an ARTIQ-8 environment. If you need a different environment,
replace the `stable` tag in the examples above with one of the following:

.. list-table::
    :header-rows: 1

    * - <TAG>
      - ARTIQ Version
    * - `old-stable`
      - ARTIQ 7
    * - `stable`
      - ARTIQ 8
    * - `latest`
      - ARTIQ Beta


.. note::
  Even if you are not interested in using atomiq, you can still use the atomiq docker image to get artiq running on
  your system without having to setup the nix packet manager

.. _install-docker-compose:

Using Docker Compose
^^^^^^^^^^^^^^^^^^^^

A convenient way to pull up the artiq master and the artiq ctlmgr is to use docker compose. You can thus replace the
above using one of the following compose files. The different configurations are for different kinds of networking, depending on your use case or preference.

.. tab-set::

  .. tab-item:: Host Network

    **Recommended for new users that do not rely on advanced networking.**

    The atomiq master can also be run in the host network. This can be easier to setup as no manual connection to a Zenoh router is necessary and no manual `hosts` entry is necessary.

    .. code-block:: yaml

        services:
          artiq-master:
            container_name: "artiq-master"
            image: registry.gitlab.com/atomiq-project/atomiq:stable
            restart: unless-stopped
            network_mode: "host"
            volumes:
              - ".:/experiment" # this should point to the folder where the device_db and the repository is located
              # - ".:/git/atomiq" # use the local atomiq installation
              # - "/path/to/some_package:/git/custom/some_package" # include custom python code

            # master command (see atomiq_master doc and manual docker section above for more options)
            command: ["python", "-m", "atomiq.frontend.atomiq_master", "--bind", "*", "-r", "/experiment/repository", "--device-db", "/experiment/device_db.py", "--heros-realm", "<realm>"]
          artiq-ctlmgr:
            container_name: "artiq-ctlmgr"
            image: registry.gitlab.com/atomiq-project/atomiq:stable
            restart: unless-stopped
            network_mode: "host"
            depends_on:
              artiq-master:
                condition: service_healthy
            command: ["artiq_ctlmgr", "--bind", "*"]


    .. note::

       The optional ``--heros-realm`` (defaults to ``heros``) is used to set the HEROS `realm <https://heros-761c0f.gitlab.io/paradigms.html#realms>`_ of all :ref:`Atomiq related HEROs <atomiq-as-hero>`.

    .. note::

       In this configuration, the ``host`` arguments of ``core_log``, ``core_moninj`` and ``core_analyzer`` in the ``device_db`` must be set to ``host: "localhost"``.

    .. admonition:: Windows
      :collapsible: closed
      :class: important

      If using Docker on Windows (unsupported) it is always necessary to connect to a zenoh peer manually with ``--heros-connect`` as WSL does not reliably support multicast (see for example this
      `Stack Overflow discussion <https://stackoverflow.com/questions/62108116/reroute-mdns-query-from-wsl-subnet-to-windows-host-subnet/62246694#62246694>`_).

  .. tab-item:: Standard Docker Networking

    Setup using the standard docker bridging network.

    You can also find this file in the `repository <https://gitlab.com/atomiq-project/atomiq/-/blob/master/docker-compose.yml>`_

    .. code-block:: yaml

        services:
          artiq-master:
            container_name: "artiq-master"
            image: registry.gitlab.com/atomiq-project/atomiq:stable
            restart: unless-stopped
            volumes:
              - ".:/experiment" # this should point to the folder where the device_db and the repository is located
              # - ".:/git/atomiq" # use the local atomiq installation
              # - "/path/to/some_package:/git/custom/some_package" # include custom python code
            ports:
              - "1380:1380" #Core device (management)
              - "1381:1381" #Core device (main)
              - "1382:1382" #Core device (analyzer)
              - "3250:3250" #notify
              - "3251:3251" #control
              - "1067:1067" #broadcast

            # master command (see atomiq_master doc and manual docker section above for more options)
            command: ["python", "-m", "atomiq.frontend.atomiq_master", "--bind", "*", "-r", "/experiment/repository", "--device-db", "/experiment/device_db.py", "--heros-connect", "tcp/<IP_OR_HOSTNAME>:7447"]
          artiq-ctlmgr:
            container_name: "artiq-ctlmgr"
            image: registry.gitlab.com/atomiq-project/atomiq:stable
            restart: unless-stopped
            ports:
              - "1066:1066" # logging
              - "3249:3249" # control
              - "1383:1383" # moninj
              - "1384:1384" # moninj
              - "1385:1385" # core-analyzer
              - "1386:1386" # core-analyzer
            depends_on:
              artiq-master:
                condition: service_healthy
            command: ["artiq_ctlmgr", "--bind", "*", "-s", "artiq-master", "-v"]

    .. important::

      For the communication between the artiq_master and the artiq_ctlmgr happens now within the automatically created
      docker network. However, this requires that you change the host for `core_moninj`, `core_log`, and `core_analyzer`
      in your `device_db.py` to the name of the artiq_ctlmgr docker service (in this case `artiq-ctlmgr`).

      Additionally, also the dashboard needs to know who is meant by `artiq-ctlmgr`. This can be achieved by adding
      :code:`--add-host artiq-ctlmgr:<IP_OR_HOSTNAME_OF_ARTIQ_CTRLMGR>` to the docker call starting the dashboard `or`
      if you installed the dashboard via :code:`pip`, you need to add a local DNS entry. On most systems this can be
      for example done by adding :code:`<IP_OR_HOSTNAME_OF_ARTIQ_CTRLMGR> artiq-ctlmgr` to :code:`/etc/hosts`, but this can vary
      depending on your distribution.

      .. admonition:: Windows
        :collapsible: closed
        :class: seealso

        Add ``127.0.0.1 artiq-ctlmgr`` to ``C:\Windows\System32\drivers\etc\hosts``.


  .. tab-item:: Macvlan (Linux only)

    You can also run the atomiq master in a macvlan network, which assigns the ``master`` and ``ctlmgr`` containers their own MAC and IP addresses.
    This is the most flexible but also the most advanced setup and is only recommended for advanced users.

    .. code-block:: yaml

        services:
          artiq-master:

            container_name: "artiq-master"
            image: registry.gitlab.com/atomiq-project/atomiq:stable
            restart: unless-stopped
            networks:
              atomiq_net:
                ipv4_address: 192.168.1.250 # adjust to your subnet
            volumes:
              - ".:/experiment" # this should point to the folder where the device_db and the repository is located
              # - ".:/git/atomiq" # use the local atomiq installation
              # - "/path/to/some_package:/git/custom/some_package" # include custom python code

            # master command (see atomiq_master doc and manual docker section above for more options)
            command: ["python", "-m", "atomiq.frontend.atomiq_master", "--bind", "*", "-r", "/experiment/repository", "--device-db", "/experiment/device_db.py"]

          artiq-ctlmgr:
            container_name: "artiq-ctlmgr"
            image: registry.gitlab.com/atomiq-project/atomiq:stable
            restart: unless-stopped
            depends_on:
              artiq-master:
                condition: service_healthy
            command: ["artiq_ctlmgr", "--bind", "*", "-s", "artiq-master", "-v"]
            networks:
              atomiq_net:
                ipv4_address: 192.168.1.251 # adjust to your subnet

        networks:
          atomiq_net:
            driver: macvlan
            driver_opts:
              parent: enp10s0 # adjust to your network interface
            ipam:
              config:
                # adjust this configuration to your used subnet.
                - subnet: "192.168.1.0/24" # your experiment subnet
                  ip_range: "192.168.1.248/29" # range of addresses the docker network can assign ips from, make sure the DHCP server in your main network does not assign ips in this range.
                  aux_addresses:
                   localhost: 192.168.1.254 # address of the shim (see below)

    To be able to connect from the host machine itself to the atomiq master, as for example for the dashboard, a shim network must be created. **If you do not plan to directly connect to the
    containers from your host machine running the containers, you can skip this step.

    A tutorial how to setup the shim can be found `here <https://blog.oddbit.com/post/2018-03-12-using-docker-macvlan-networks/#host-access>`_. If you use `Network Manager <https://networkmanager.dev/>`_
    you can configure the shim with the following command (adjust the interface name (here ``enp10s0``) and the ip addresses to your setup):

    .. code-block:: bash

      sudo nmcli connection add type macvlan ifname atomiq-shim mode bridge dev enp10s0 -- +ipv4.method manual ipv4.addresses 192.168.1.254/32 +ipv4.routes 192.168.1.248/29`

    .. note::

       In this configuration, the ``host`` arguments of ``core_log``, ``core_moninj`` and ``core_analyzer`` in the ``device_db`` must be set to the ip of the ``artiq_ctlmgr`` address. In the example this would be ``host: "192.168.1.251"``.



Add Custom Code/Software to the ARTIQ Environment
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

Extending the available software in the docker container of the artiq_master is required if
your experiment scripts need to make use of some custom tools. Depending one your requirements
this can be solved in two ways (or both at the same time):

1. **Add your own python code into the atomiq container**
   To add your own python code (that does not depend on any external libraries) you can just create a bind mount
   from the local directory containing your custom code to the `/git/custom` directory in the container. This
   directory is in the python path and every python file in this folder can be imported in python.

2. **Add third party packages or non-python tools to the artiq container**
   To achieve this you can just create your own docker image that derives from the atomiq docker image.
   Then simply start your atomiq_master from your custom, extended image rather than from
   `registry.gitlab.com/atomiq-project/atomiq:stable`.

   The following simple example :code:`dockerfile` extends the image by the `scipy` python library

   .. code-block:: Docker

      FROM registry.gitlab.com/atomiq-project/atomiq:stable
      RUN pip install --break-system-packages scipy


   `Here <https://docs.docker.com/build/building/base-images/>`_ you can learn more on how to extend docker images.

   .. Warning::

      The docker image uses alpine linux as a base, which uses the musl c-library.
      If you need to use device libraries compiled for glibc, we recommend running them outside the docker container
      and connecting them with `HEROS <https://heros-761c0f.gitlab.io/>`_.


Changing the Atomiq Code in the Running system
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

For development purposes, it is useful to be able to modify the atomiq and the artiq code. To achieve this,
clone a local copy of atomiq/artiq from gitlab/github and use docker bind mounts to overlay the paths `/git/atomiq`
or `/git/artiq` with your locally cloned copy. Make sure that the versions you cloned fits your system.


Installation via Nix
--------------------

.. Warning::

   We do not support this method anymore since it is virtually impossible to keep the nix flake working without a lot of effort.
   Chances are high that the method below will not work anymore.
   Please use the docker method above whenever possible!

#. Acquire a working `nix` installation and enable `nix flakes``. You can follow the official
   `ARTIQ docs <https://m-labs.hk/artiq/manual/installing.html#installing-via-nix-linux>`_.

Atomiq provides a nix flake file that can either be run directly or called from an existing ARTIQ nix flake file.

Atomiq as the Main Flake
^^^^^^^^^^^^^^^^^^^^^^^^
This method is recommended for new setups.

#. Install atomiq and ARTIQ by installing `flake.nix` from within this repository

 .. code-block:: bash

    $ cd atomiq
    $ nix develop .#dev-<branch>
    # some installation happening
    $ which artiq_master
    /nix/store/<__i_am_a_hash__>-python<version>-env/bin/artiq_master
    $ python -c "import atomiq,artiq; print(dir())"
    [(...), 'artiq', 'atomiq']

 Here, branch is one of the following, depending on which version of ARTIQ you are using

 .. list-table::
    :header-rows: 1

    * - <branch>
      - ARTIQ Version
    * - `old_stable`
      - ARTIQ 7
    * - `stable`
      - ARTIQ 8
    * - `beta`
      - ARTIQ Beta

 .. note::
    Local development of atomiq needs the atomiq sources to be linked into the nix environment. A shell doing this can be entered by running

    .. code-block:: bash

      $ nix develop .#localdev-<branch>


Embed in Existing Flake
^^^^^^^^^^^^^^^^^^^^^^^
1. If you have an existing flake you want to use, you can include atomiq as an input. Add the following line to the top of your flake file:

  .. code-block:: nix


      inputs.artiq = "git+https://github.com/m-labs/artiq.git?<ref>";
      inputs.atomiq.url = "git+https://gitlab.com/atomiq-project/atomiq";
      inputs.atomiq.inputs.artiqpkgs_<branch>.follows = "artiq";


  Here, `<branch>` is the atomiq branch corresponding to the different artiq versions `<ref>` as below:

  .. list-table::
    :header-rows: 1

    * - <branch>
      - <ref>
    * - `old_stable`
      - `?ref=release-7`
    * - `stable`
      - `?ref=release-8`
    * - `beta`
      -

2. And add it to the outputs:

  .. code-block:: nix

      outputs = { self, artiq, extrapkg, atomiq}:

3. Inside your derivation or shell, you can then load atomiq by adding it to the Python package list

  .. code-block:: nix

    (pkgs.python3.withPackages(ps: [
      atomiq.packages.x86_64-linux.atomiq_artiq_<branch>
    ]))

  where `<branch>` is one of the branch names as in the table above.

  .. warning::
    Atomiq provides the ARTIQ package as a propagated dependency. To avoid a
    collision error, the base `artiq` package must be removed from the Python
    package list

A sample `flake.nix` file can be found in :ref:`this example <Nix example>`

.. _install-dashboard-only:

Installing Client/Dashboard Only
--------------------------------

If you want to only run the dashboard (including the atomiq modifications) or the artiq client for example
on an auxiliary lab computer or your office pc, you can simply install ARTIQ and atomiq in a python virtual
environment by running:

.. note::

  We recommend using `uv <https://docs.astral.sh/uv/>`_ to maintain an enclosed python environment.

.. tab-set::

   .. tab-item:: uv

      .. code-block::

        # Create a new virtual environment
        uv venv

        # Install required packages
        uv pip install git+https://gitlab.com/atomiq-project/atomiq[gui-stable]


   .. tab-item:: other

      .. code-block:: bash

        # Create a new virtual environment
        python -m venv /path/to/your/venv

        # Activate the virtual environment
        source /path/to/your/venv/bin/activate

        # Install required packages
        pip install git+https://gitlab.com/atomiq-project/atomiq[gui-stable]


You can specify the ARTIQ version by adjusting the branch. Currently, we support `release-7` ``[gui-old-stable]``, `release-8` ``[gui-stable]``, and `master` ``[gui-beta]``.

Now, the dashboard can be launched by running:

.. tab-set::

   .. tab-item:: uv

      .. code-block::

        uv run atomiq_dashboard


   .. tab-item:: other

      .. code-block:: bash

        source /path/to/your/venv/bin/activate && atomiq_dashboard

The vanilla dashboard can be launched analogously:

.. tab-set::

   .. tab-item:: uv

      .. code-block::

        uv run artiq_dashboard


   .. tab-item:: other

      .. code-block:: bash

         source /path/to/your/venv/bin/activate && artiq_dashboard

In both cases, command line arguments can be specified as usual.

.. important::

   Most of the other ARTIQ tools (e.g. ``artiq_master``) will not work with the virtual environment only.