Infinite-ISP Tutorial: Contributing to Infinite-ISP
Infinite-ISP, an open-source project, provides a versatile platform for processing raw images. In this tutorial we’ll discuss how you can become a valuable contributor to our project by suggesting new algorithms or tuning/ improving the existing ones.
Before we dive in, make sure you have Infinite-ISP cloned and configured. You can find installation instructions here. Create a new branch using the following command.
Module class structure
All modules are implemented as classes with some common class attributes and methods and can be found in the ./modules directory.
Each module takes the raw image, sensor specifications and module parameters from the config file. All parameters are encapsulated within python dictionaries. Some common class attributes and methods along with their brief description are as below:
Common class attributes among different modules
|Class Attribute name||Description||Type|
|Image||Raw image from sensor or output of the previous module||ndarray with dtype uint16|
|sensor_info||Sensor specifications from config||dict|
|parm_<module_name>||Module specific parameters from config file||dict|
|platform||contains configuration parameters that are not part of the ISP pipeline but helps in pipeline execution and debugging||dict|
|is_enable||Flag to enable/disable module||boolean|
|is_debug||Flag to printing detailed logs for debugging purposes||boolean|
|is_save||Flag to save module output||boolean|
Common methods among different modules:
|__init__||To initialize class attributes||Img, sensor_info, module parameters, platform(for some modules)||None|
|apply_<module_name>||Method with algorithm implementation||Varies across modules||module output ndarray with dtype uint16|
|save||To save module output||None||None|
|execute||Calls apply method to execute a module||None||module output array with dtype uint16|
To add a new module to Infinite-ISP, first create a Python file in the “./modules” directory. A good place to start is by replicating an existing module file and then altering it as needed. You’ll mainly need to plug your algorithm under the apply method keeping execute and save methods the same. While implementing your module, ensure that moodule input and output are uint16 ndarrays for seamless integration with infinite-ISP pipeline.
Plugging your module in Infinite-ISP
Position of your module in the pipeline using the context in which your module operates and interacts with various color spaces. The image below summarizes the working of infinite-ISP emphasizing transitions between color spaces. Add your module block in the default config file (placed at ./config directory) at the respective position. This step is not necessary but helps as a reminder of the pipeline sequence.
Next, modify the infinite_isp.py file to include the newly implemented module as a part of the infinite-ISP object. Following modifications are required:
- Import your module
- Modify the load config method to load your module parameters from config
- Modify the run_pipeline method:
- Place your module block in the pipeline.
- Pass the output of the previous module as input to the new module.
- Similarly, pass the output of this new module to the next module in the pipeline.
Now that you have successfully integrated your module into the pipeline, let’s see it in action. Enable your module in the config file and run infinite_isp.py file. You can use the pipeline output or module output (using is_save flag) for visual assessment of the processed image or use some image quality metrics to compare results.
All the best with your debugging endeavors!!
Once you have resolved all errors and the module is working properly in combination with other modules you can work on the code linting to write clean and readable code. Pylint library is used for uniform code styling across the project. Install Pylint and pylint-runner using the commands below to see your score.
You can also use black library for a quick fix for white spaces etc.
Refer to the suggested changes by pylint to style your code. You can generate a pull request on github once you have achieved a score 10.