Skip to content
GitLab
Commit 6b456cdd authored by Arnaud WATLET's avatar Arnaud WATLET
Browse files

Updates software doc

parent 391f0a78
master 144-create-testing-framework-for-tests-automation v2024_rc v2024.0.34 v2024.0.33 v2024.0.32 v2024.0.31 v2024.0.30 v2024.0.29 v2024.0.28 v2024.0.27 v2024.0.26 v2024.0.25 v2024.0.24 v2024.0.23 v2024.0.22 v2024.0.21 v2024.0.20 v2024.0.19 v2024.0.18 v2024.0.17 v2024.0.16 v2024.0.15 v2024.0.14 v2024.0.13 v2024.0.12 v2024.0.11 v2024.0.10 v2024.0.9 v2024.0.8 v2024.0.7 v2024.0.0
No related merge requests found
Hide whitespace changes
Inline Side-by-side
Showing
with 327 additions and 242 deletions
+327 -242
doc/source/source_rst/software.rst
+ 7
− 242
View file @ 6b456cdd
********************************************
Software and operation Software and operation
******************************************** #####################
.. warning:: .. warning::
**OhmPi is a participative project open to all, it requires skills in electronics and to respect the safety rules. OhmPi must be assembled in a professional context and by people competent in electronics. The OhmPi team cannot be held responsible for any material or human damage which would be associated with the use or the assembly of OhmPi. The OhmPi team cannot be held responsible if the equipment does not work after assembly.** **OhmPi is a participative project open to all, it requires skills in electronics and to respect the safety rules. OhmPi must be assembled in a professional context and by people competent in electronics. The OhmPi team cannot be held responsible for any material or human damage which would be associated with the use or the assembly of OhmPi. The OhmPi team cannot be held responsible if the equipment does not work after assembly.**
This section details describes the Python software and how to interact with an OhmPi instrument.
System architecture .. toctree::
-------------------
The OhmPi V2023 software is designed around a new architecture whose main components are summarized in the figure below. Software architecture <software/architecture>
Installation <software/installation>
Operating the instrument <software/operations>
How to contribute <software/developments>
.. figure:: ../img/architecture.png
Software architecture of OhmPi V2023.
The general system configuration is defined in the `config.py` file covered in the `Configuration file`_ section.
The acquisition settings (i.e. injection duration, stacks...) are defined in a separate JSON file (default: ohmpi_settings.json).
The central software component is the ohmpi.py file that contains the OhmPi class that interacts with the hardware. Other python files include utils and handlers (see the `Loggers`_ section for more details).
A communication layer (I/O interface) on top of OhmPi allows for different user interfaces depending on the use cases (see `Interfaces and applications`_).
Loggers
-------
Loggers have been introduced in this release. They use the excellent logging python package.
Specific handlers have been implemented for running with ohmpi.py (one for logging to an mqtt broker (see `MQTT interface`_ for more details) and one for creating zipped rotated logs on disk).
Two loggers have been defined. The first one is dedicated to log operations execution. It is named exec_logger. The second one, named data_logger, is dedicated to log data. A third one is planned to log the state of health (SOH) of the system in a future version.
By default, logs are written to the console (print-like), stored locally in files (a zip is created after some time i.e. every day and/or when the size of the log exceeds a maximum size) and sent to an MQTT broker. Different logging levels may be defined for the different logs and handlers in the `Configuration file`_.
Advanced users may write new handlers and edit the `setup_loggers.py` file to customize the logging mechanisms to their needs.
Configuration file
------------------
The configuration of the OhmPi file `config.py` allows to configure the OhmPi.
A default version of `config.py` is provided in the repository.
This file should be edited to customize the configuration following the user's needs and preferences.
The configuration includes setting the logging level desired for the different loggers and handlers, setting the mqtt broker(s) used for logging and control of the OhmPi and defining the options used for MQTT communication (i.e. username, password, security options...)
One should make sure to understand the parameters before altering them. It is also recommended to keep a copy of the default configuration.
Interfaces and applications
---------------------------
Different interfaces can be used to interact with the OhmPi.
Available interfaces are:
- `Web interface`_ (=HTTP interface): run in bash: `bash run_http_interface.sh`
- Python API: import the OhmPi class from Python script: `from ohmpi import OhmPi` (see `Python interface`_)
- MQTT: IoT messaging through a broker (see `MQTT interface`_)
Web interface
.............
This is a user friendly graphical interface for new users as well as running quick and easy acquisitions.
The Raspberry Pi of the OhmPi is used as a Wi-Fi Access Point (AP) and runs
a small webserver to serve the 'index.html' interface. Using a laptop or
a mobile phone connected to the Wi-Fi of the Raspberry Pi, one can see this
interface, upload sequences, change parameters, run a sequence and download data.
To configure the Raspberry Pi to act as an access point and run
the webserver automatically on start, see instructions on `raspap.com <https://raspap.com/>`_ and in 'runOnStart.sh'.
Once configured, the webserver should start by itself on start and once
connected to the Pi, the user can go to `10.3.141.1:8080 <http://10.3.141.1:8080>`_
to access the interface.
.. figure:: ../img/http-interface-pseudo-section.png
Web interface with its interactive pseudo-section.
.. figure:: ../img/http-interface-evolution.png
Evolution of quadrupole apparent resistivity with time.
.. figure:: ../img/http-interface-rs.png
Contact resistance check.
Python interface
................
This interface offers a more direct access to the software components especially well suited for testing or automation on the Raspberry Pi.
By importing the `OhmPi` class from the ohmpi.py, one can control the OhmPi using interactive IPython.
Typically, it involves using the terminal or an Python IDE such as Thonny on the Raspberry Pi. One can also connect using
ssh and run the Python interface (see PuTTY on Windows or ssh command on macOS/Linux).
To access the Python API, make sure the file ohmpi.py is in the same
directory as where you run the commands/script. The file ohmpi.py can
be found on the OhmPi gitlab repository. We recommend downloading the
entire repository as ohmpi.py import other .py files and default configuration
files (.json and .py).
.. code-block:: python
:caption: Example of using the Python API to control OhmPi
import os
import numpy as np
import time
os.chdir("/home/pi/OhmPi")
from ohmpi import OhmPi
### Define object from class OhmPi
k = OhmPi() # this loads default parameters from the disk
### Default parameters can also be edited manually
k.settings['injection_duration'] = 0.5 # injection time in seconds
k.settings['nb_stack'] = 1 # one stack is two half-cycles
k.settings['nbr_meas'] = 1 # number of time the sequence is repeated
### Update settings if needed
k.update_settings({"injection_duration":0.2})
### Set or load sequence
k.sequence = np.array([[1,2,3,4]]) # set numpy array of shape (n,4)
# k.set_sequence('1 2 3 4\n2 3 4 5') # call function set_sequence and pass a string
# k.load_sequence('ABMN.txt') # load sequence from a local file
### Run contact resistance check
k.rs_check()
### Run sequence (synchronously - it will wait that all
# sequence is measured before returning the prompt
k.run_sequence()
# k.run_sequence_async() # sequence is run in a separate thread and the prompt returns immediately
# time.sleep(2)
# k.interrupt() # kill the asynchron sequence
### Run multiple sequences at given time interval
k.settings['nb_meas'] = 3 # run sequence three times
k.settings['sequence_delay'] = 100 # every 100 s
k.run_multiple_sequences() # asynchron
# k.interrupt() # kill the asynchron sequence
### Single measurement can also be taken with
k.switch_mux_on([1, 4, 2, 3])
k.run_measurement() # use default acquisition parameters
k.switch_mux_off([1, 4, 2, 3]) # don't forget this! risk of short-circuit
### Custom or adaptative argument, see help(k.run_measurement)
k.run_measurement(nb_stack=4, # do 4 stacks (8 half-cycles)
injection_duration=2, # inject for 2 seconds
autogain=True) # adapt gain of ADS to get good resolution
MQTT interface
..............
This is an interface designed for an advanced remote usage of the OhmPi such as remote automation, data consumption by multiple processes and interaction with other sensors in the scope of a monitoring. It is based on the MQTT protocol, designed for the Internet of Things (IoT), to interact with the OhmPi.
This option allows interacting remotely with a single OhmPi, a network of OhmPis, as well as auxiliary instruments and sensors. The communication is based on a publish/subscribe approach and involves a MQTT broker.
An example of MQTT broker that can be used is `Mosquitto <https://mosquitto.org/>`_. Depending on the monitoring needs, an MQTT broker can be set up locally on the Raspberry Pi, on a local network or any remote server reachable through the net. A local Mosquitto broker can be set up and enabled to run as a service on the OhmPi using the bash script install_local_mqtt_broker.sh.
MQTT messages include logging messages from the OhmPi and commands sent to the OhmPi. These messages can be examined easily using a third party software such as `MQTT Explorer <http://mqtt-explorer.com/>`_.
Commands sent on the broker are received by the ohmpi.py script that runs on the OhmPi (make sure ohmpi.py starts on reboot) and further processed.
MQTT commands are sent in JSON format following the Python API with kwargs as illustrated below:
.. code-block:: json
:caption: Updating acquisition settings.
{
"cmd_id": "3fzxv121UITwGjWYgcz4xw",
"cmd": "update_settings", Depending on the experiment needs, MQTT brokers can be set up locally on the Raspberry Pi or on a local or remote server.
"kwargs": {
"config": {
"nb_meas": 2,
"nb_electrodes": 10,
"nb_stack": 2,
"injection_duration": 2,
"sequence_delay": 100
}
}
}
.. code-block:: json
:caption: Check contact resistances
{
"cmd_id": "3fzxv121UITwGjWYgcz4xw",
"cmd": "rs_check",
}
.. code-block:: json
:caption: Running a sequence.
{
"cmd_id": "3fzxv121UITwGjWYgcz4Yw",
"cmd": "run_sequence",
}
.. code-block:: json
:caption: Running same sequence multiple times (nb_meas).
{
"cmd_id": "3fzxv121UITwGjWYgcz4Yw",
"cmd": "run_multiple_sequences",
}
.. code-block:: json
:caption: Interrupt current acquisition.
{
"cmd_id": "3fzxv121UITwGjWYgcz4xw",
"cmd": "interrupt",
}
Custom processing of messages and tailor-made dashboards for monitoring experiments may be designed using a browser-based flow editor such as `Node-red <http://mqtt-explorer.com/>`_.
This may help designing complex IoT experiments and monitoring systems in which OhmPi is a component.
Examples incorporating execution commands and data outputs from OhmPi can be found in the OhmPi examples. Once Node-RED is installed on the OhmPi, these examples can be accessed separately by running a command in the console such as :
.. code-block:: console
node-red basic_ohmpi_flows_node-red.json
These examples may require installing some additional node packages in order to work properly. This can be done in the `Palette Manager <https://nodered.org/docs/user-guide/editor/palette/manager> within Node-RED.
.. figure:: ../img/node-red_flow.png
Example flow in node-red to interact with an OhmPi.
.. figure:: ../img/node-red_interface_control.png
Example of a dashboard UI created with node-red to interact with an OhmPi - control tab.
.. figure:: ../img/node-red_interface_data.png
Example of a dashboard UI created with node-red to interact with an OhmPi - data visualization tab.
For more documentation dedicated to node-red, please refer to the Node-red `cookbooks <https://cookbook.nodered.org/>`_.
doc/source/source_rst/software/architecture.rst 0 → 100644
+ 52
− 0
View file @ 6b456cdd
Software architecture
*******************
The OhmPi V2024 software has been completely re-structured to enable increased flexibility for both users and developers. The software is based on an object-oriented module with a class exposing the OhmPi
functionalities used to interact with the OhmPi instrument via a web interface, IoT
communication protocols (e.g. MQTT) and/or directly through the Python API.
.. figure:: ../img/architecture.png
Software architecture of OhmPi V2024.
The software is organised in several modules describing the system in a hierarchy including three levels of description and
operation of the OhmPi.
Hardware components
===================
On the base level, the five main hardware components are represented by distinct classes exposing the
components atomic functionalities. Theses classes are abstract classes in order to provide a common
interface for different implementations of a component. From these abstract classes concrete classes
are implemented representing the default properties, actual capabilities and ways to interact with the
physical modules or boards.
Improving an existing hardware component or introducing a new design may be desirable in order to,
e.g. reduce costs, improve performance, adapt measurement range to specific applications, or
incorporate easily available electronic components. It is at this level that software developments are
mainly expected to occur following updates on the hardware. The component class should expose the
minimal functionalities required by the hardware system (see below) for this type of component.
Hardware system
===============
On the medium level, the OhmPiHardware class provides a mean to assemble and operate the
acquisition system. The methods of this class orchestrate atomic operations of the system components
in order to expose basic system functionalities such as cross-MUX switching, square wave voltage
injection or full waveform voltage and current reading during injection cycles. These functionalities
are implemented using synchronization mechanisms between threads in order to insure that each
component keeps in step with the others.
The whole system is described in a configuration file listing the hardware components and versions
used. Through a dynamic import mechanism the modules containing the classes corresponding with
the physical hardware modules of a particular OhmPi system are instantiated and associated with the
system object instantiated from the OhmPiHardware class. In this way, it is relatively simple to build
customised systems once the concrete classes describing the system components have been written.
This part of the software architecture should remain stable if the overall system functionalities do not
evolve. However, the introduction of new functionalities at the system level or radical changes in the
way the components work together will require adaptations at this level.
Acquisition
===========
On the top level, the OhmPi class (in ohmpi/ohmpi.py) includes all the higher-level methods and properties allowing to
operate the system (e.g. acquire measurement sequences). The OhmPi class exposes the user-oriented
API, generates logs and handles IoT messages. Generic users are expected to interact with the system
through these higher-level functionalities, which are designed to remain as stable as possible while the
hardware evolves. Only the introduction of new end-user functionalities should imply new
developments at this level. See
\ No newline at end of file
doc/source/source_rst/software/developments.rst 0 → 100644
+ 0
− 0
View file @ 6b456cdd
doc/source/source_rst/software/installation.rst 0 → 100644
+ 39
− 0
View file @ 6b456cdd
Installation
************
Step 1: Connect to the Raspberry Pi
===================================
Step 2: Clone the OhmPi project
===============================
Step 3: Run the installation script
===================================
Simply navigate to the OhmPi folder and run the following command on the terminal:
.. code-block:: bash
$ .install.sh
The install script first creates an python virtual environment called "ohmpy" in which all dependencies will be installed. Dependecies are specified in requirements.txt
When the installation is completed check that requirements are met using pip list.
It then installs a local MQTT broker which will be used to centralise all the communication between the hardware, software and interfaces.
It also properly configures the I2C buses on the raspberry Pi.
When the installation is performed, we need to add the OhmPi module to the environment by editing the PYTHONPATH or editing the .bashrc file as follows:
.. code-block:: bash
$ nano ~/.bashrc
And add the following line:
.. code-block:: bash
$export PYTHONPATH=$PYTHONPATH:/home/<username>/OhmPi
Step 4: Activate the ohmpy virtual environment
==============================================
Before operating the instrument, we need to activate the ohmpy virtual environment with the following command:
.. code-block:: bash
$ cd ~/OhmPi
$ source ohmpy/bin/activate
If you need to leave the virtual environment, simply type:
.. code-block:: bash
$ deactivate
\ No newline at end of file
doc/source/source_rst/software/operations.rst 0 → 100644
+ 229
− 0
View file @ 6b456cdd
Configuration file
==================
The configuration of the OhmPi file `config.py` allows to configure the OhmPi.
A default version of `config.py` is provided in the repository.
This file should be edited to customize the configuration following the user's needs and preferences.
The configuration includes setting the logging level desired for the different loggers and handlers, setting the mqtt broker(s) used for logging and control of the OhmPi and defining the options used for MQTT communication (i.e. username, password, security options...)
One should make sure to understand the parameters before altering them. It is also recommended to keep a copy of the default configuration.
Interfaces and applications
===========================
Different interfaces can be used to interact with the OhmPi.
Available interfaces are:
- `Web interface`_ (=HTTP interface): run in bash: `bash run_http_interface.sh`
- Python API: import the OhmPi class from Python script: `from ohmpi import OhmPi` (see `Python interface`_)
- MQTT: IoT messaging through a broker (see `MQTT interface`_)
Web interface
.............
This is a user friendly graphical interface for new users as well as running quick and easy acquisitions.
The Raspberry Pi of the OhmPi is used as a Wi-Fi Access Point (AP) and runs
a small webserver to serve the 'index.html' interface. Using a laptop or
a mobile phone connected to the Wi-Fi of the Raspberry Pi, one can see this
interface, upload sequences, change parameters, run a sequence and download data.
To configure the Raspberry Pi to act as an access point and run
the webserver automatically on start, see instructions on `raspap.com <https://raspap.com/>`_ and in 'runOnStart.sh'.
Once configured, the webserver should start by itself on start and once
connected to the Pi, the user can go to `10.3.141.1:8080 <http://10.3.141.1:8080>`_
to access the interface.
.. figure:: ../img/http-interface-pseudo-section.png
Web interface with its interactive pseudo-section.
.. figure:: ../img/http-interface-evolution.png
Evolution of quadrupole apparent resistivity with time.
.. figure:: ../img/http-interface-rs.png
Contact resistance check.
Python interface
................
This interface offers a more direct access to the software components especially well suited for testing or automation on the Raspberry Pi.
By importing the `OhmPi` class from the ohmpi.py, one can control the OhmPi using interactive IPython.
Typically, it involves using the terminal or an Python IDE such as Thonny on the Raspberry Pi. One can also connect using
ssh and run the Python interface (see PuTTY on Windows or ssh command on macOS/Linux).
To access the Python API, make sure the file ohmpi.py is in the same
directory as where you run the commands/script. The file ohmpi.py can
be found on the OhmPi gitlab repository. We recommend downloading the
entire repository as ohmpi.py import other .py files and default configuration
files (.json and .py).
.. code-block:: python
:caption: Example of using the Python API to control OhmPi
import os
import numpy as np
import time
os.chdir("/home/pi/OhmPi")
from ohmpi import OhmPi
### Define object from class OhmPi
k = OhmPi() # this loads default parameters from the disk
### Default parameters can also be edited manually
k.settings['injection_duration'] = 0.5 # injection time in seconds
k.settings['nb_stack'] = 1 # one stack is two half-cycles
k.settings['nbr_meas'] = 1 # number of time the sequence is repeated
### Update settings if needed
k.update_settings({"injection_duration":0.2})
### Set or load sequence
k.sequence = np.array([[1,2,3,4]]) # set numpy array of shape (n,4)
# k.set_sequence('1 2 3 4\n2 3 4 5') # call function set_sequence and pass a string
# k.load_sequence('ABMN.txt') # load sequence from a local file
### Run contact resistance check
k.rs_check()
### Run sequence (synchronously - it will wait that all
# sequence is measured before returning the prompt
k.run_sequence()
# k.run_sequence_async() # sequence is run in a separate thread and the prompt returns immediately
# time.sleep(2)
# k.interrupt() # kill the asynchron sequence
### Run multiple sequences at given time interval
k.settings['nb_meas'] = 3 # run sequence three times
k.settings['sequence_delay'] = 100 # every 100 s
k.run_multiple_sequences() # asynchron
# k.interrupt() # kill the asynchron sequence
### Single measurement can also be taken with
k.switch_mux_on([1, 4, 2, 3])
k.run_measurement() # use default acquisition parameters
k.switch_mux_off([1, 4, 2, 3]) # don't forget this! risk of short-circuit
### Custom or adaptative argument, see help(k.run_measurement)
k.run_measurement(nb_stack=4, # do 4 stacks (8 half-cycles)
injection_duration=2, # inject for 2 seconds
autogain=True) # adapt gain of ADS to get good resolution
MQTT interface
..............
This is an interface designed for an advanced remote usage of the OhmPi such as remote automation, data consumption by multiple processes and interaction with other sensors in the scope of a monitoring. It is based on the MQTT protocol, designed for the Internet of Things (IoT), to interact with the OhmPi.
This option allows interacting remotely with a single OhmPi, a network of OhmPis, as well as auxiliary instruments and sensors. The communication is based on a publish/subscribe approach and involves a MQTT broker.
An example of MQTT broker that can be used is `Mosquitto <https://mosquitto.org/>`_. Depending on the monitoring needs, an MQTT broker can be set up locally on the Raspberry Pi, on a local network or any remote server reachable through the net. A local Mosquitto broker can be set up and enabled to run as a service on the OhmPi using the bash script install_local_mqtt_broker.sh.
MQTT messages include logging messages from the OhmPi and commands sent to the OhmPi. These messages can be examined easily using a third party software such as `MQTT Explorer <http://mqtt-explorer.com/>`_.
Commands sent on the broker are received by the ohmpi.py script that runs on the OhmPi (make sure ohmpi.py starts on reboot) and further processed.
MQTT commands are sent in JSON format following the Python API with kwargs as illustrated below:
.. code-block:: json
:caption: Updating acquisition settings.
{
"cmd_id": "3fzxv121UITwGjWYgcz4xw",
"cmd": "update_settings", Depending on the experiment needs, MQTT brokers can be set up locally on the Raspberry Pi or on a local or remote server.
"kwargs": {
"config": {
"nb_meas": 2,
"nb_electrodes": 10,
"nb_stack": 2,
"injection_duration": 2,
"sequence_delay": 100
}
}
}
.. code-block:: json
:caption: Check contact resistances
{
"cmd_id": "3fzxv121UITwGjWYgcz4xw",
"cmd": "rs_check",
}
.. code-block:: json
:caption: Running a sequence.
{
"cmd_id": "3fzxv121UITwGjWYgcz4Yw",
"cmd": "run_sequence",
}
.. code-block:: json
:caption: Running same sequence multiple times (nb_meas).
{
"cmd_id": "3fzxv121UITwGjWYgcz4Yw",
"cmd": "run_multiple_sequences",
}
.. code-block:: json
:caption: Interrupt current acquisition.
{
"cmd_id": "3fzxv121UITwGjWYgcz4xw",
"cmd": "interrupt",
}
Custom processing of messages and tailor-made dashboards for monitoring experiments may be designed using a browser-based flow editor such as `Node-red <http://mqtt-explorer.com/>`_.
This may help designing complex IoT experiments and monitoring systems in which OhmPi is a component.
Examples incorporating execution commands and data outputs from OhmPi can be found in the OhmPi examples. Once Node-RED is installed on the OhmPi, these examples can be accessed separately by running a command in the console such as :
.. code-block:: console
node-red basic_ohmpi_flows_node-red.json
These examples may require installing some additional node packages in order to work properly. This can be done in the `Palette Manager <https://nodered.org/docs/user-guide/editor/palette/manager> within Node-RED.
.. figure:: ../img/node-red_flow.png
Example flow in node-red to interact with an OhmPi.
.. figure:: ../img/node-red_interface_control.png
Example of a dashboard UI created with node-red to interact with an OhmPi - control tab.
.. figure:: ../img/node-red_interface_data.png
Example of a dashboard UI created with node-red to interact with an OhmPi - data visualization tab.
For more documentation dedicated to node-red, please refer to the Node-red `cookbooks <https://cookbook.nodered.org/>`_.
Loggers
-------
Loggers have been introduced in this release. They use the excellent logging python package.
Specific handlers have been implemented for running with ohmpi.py (one for logging to an mqtt broker (see `MQTT interface`_ for more details) and one for creating zipped rotated logs on disk).
Two loggers have been defined. The first one is dedicated to log operations execution. It is named exec_logger. The second one, named data_logger, is dedicated to log data. A third one is planned to log the state of health (SOH) of the system in a future version.
By default, logs are written to the console (print-like), stored locally in files (a zip is created after some time i.e. every day and/or when the size of the log exceeds a maximum size) and sent to an MQTT broker. Different logging levels may be defined for the different logs and handlers in the `Configuration file`_.
Advanced users may write new handlers and edit the `setup_loggers.py` file to customize the logging mechanisms to their needs.
Supports Markdown
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment