Infinite-ISP Tutorial: Contributing to Infinite-ISP

Introduction

Infinite-ISP, an open-source project, provides a versatile platform for processing raw images. In this tutorial we’ll discuss how you can become a valuable contributor to our project by suggesting new algorithms or tuning/ improving the existing ones.

Before we dive in, make sure you have Infinite-ISP cloned and configured. You can find installation instructions here. Create a new branch using the following command.

git checkout -b branch_name
Module class structure

All modules are implemented as classes with some common class attributes and methods and can be found in the ./modules directory.

Each module takes the raw image, sensor specifications and module parameters from the config file. All parameters are encapsulated within python dictionaries. Some common class attributes and methods along with their brief description are as below:

Common class attributes among different modules

Class Attribute name Description Type
Image Raw image from sensor or output of the previous module ndarray with dtype uint16
sensor_info Sensor specifications from config dict
parm_<module_name> Module specific parameters from config file dict
platform contains configuration parameters that are not part of the ISP pipeline but helps in pipeline execution and debugging dict
is_enable Flag to enable/disable module boolean
is_debug Flag to printing detailed logs for debugging purposes boolean
is_save Flag to save module output boolean

Common methods among different modules:

Method Name Description Arguments Output
__init__ To initialize class attributes Img, sensor_info, module parameters, platform(for some modules) None
apply_<module_name> Method with algorithm implementation Varies across modules module output ndarray with dtype uint16
save To save module output None None
execute Calls apply method to execute a module None module output array with dtype uint16
Module implementation

To add a new module to Infinite-ISP, first create a Python file in the “./modules” directory. A good place to start is by replicating an existing module file and then altering it as needed. You’ll mainly need to plug your algorithm under the apply method keeping execute and save methods the same. While implementing your module, ensure that moodule input and output are uint16 ndarrays for seamless integration with infinite-ISP pipeline.

Plugging your module in Infinite-ISP

Position of your module in the pipeline using the context in which your module operates and interacts with various color spaces. The image below summarizes the working of infinite-ISP emphasizing transitions between color spaces. Add your module block in the default config file (placed at ./config directory) at the respective position. This step is not necessary but helps as a reminder of the pipeline sequence.

Next, modify the infinite_isp.py file to include the newly implemented module as a part of the infinite-ISP object. Following modifications are required:

  • Import your module
  • Modify the load config method to load your module parameters from config
  • Modify the run_pipeline method:
    • Place your module block in the pipeline.
    • Pass the output of the previous module as input to the new module.
    • Similarly, pass the output of this new module to the next module in the pipeline.
Module Verification

Now that you have successfully integrated your module into the pipeline, let’s see it in action. Enable your module in the config file and run infinite_isp.py file. You can use the pipeline output or module output (using is_save flag) for visual assessment of the processed image or use some image quality metrics to compare results.

All the best with your debugging endeavors!!

Code formatting

Once you have resolved all errors and the module is working properly in combination with other modules you can work on the code linting to write clean and readable code. Pylint library is used for uniform code styling across the project. Install Pylint and pylint-runner using the commands below to see your score.

pip install pylint
pip install pylint_runner
pylint_runner

You can also use black library for a quick fix for white spaces etc.

pip install black
black ./modules/<new_module>.py

Refer to the suggested changes by pylint to style your code. You can generate a pull request on github once you have achieved a score 10.