Infinite-ISP Tutorial: Getting Started

Welcome to our open-source project on image signal processor (ISP) pipeline known as Infinite-ISP. This cutting-edge system consists of a wide range of camera pipeline components that have been meticulously designed to convert sensor-captured RAW photos into bright RGB output. This project aims to centralize all the open-source ISP development to a single place enabling all the ISP developers to have a single platform to contribute. Follow along with this tutorial as we study its practical uses and explore its possibilities.

In this tutorial, you will learn:

  • How to set up Infinite-ISP repository
  • Infinite-ISP repository structure
  • How to process a raw file using Infinite-ISP pipeline object

Installation

Pre-Requisites

Before we start, ensure you have python installed on your computer and an active github account . I’m using Python 3.9.12 for this tutorial.

Github repository: Reference Model

Let’s Begin!

To clone the Infinite-ISP repository (repo) on your local computer, open your terminal and go to the directory where you want to place this cloned repo. Copy and paste the following command and press Enter.

git clone https://github.com/10xEngineersTech/Infinite-ISP_ReferenceModel

Dependencies

Use the command below in your terminal to install all dependencies using pip.

pip install -r requirements.txt

  • fxpmath
  • matplotlib
  • numpy
  • PyYAML
  • rawpy
  • scipy
  • tqdm

Getting Started

Once you have successfully set up the repo locally, you can start processing your raw data!

Repository Structure

Open the cloned directory in a code editor of your choice (I’m using VScode). The contents of the cloned directory as viewed in the explorer tab are as follows:

Infinite-ISP directory with boxed main file

Isp_pipeline.py is the main file that uses the infinite-ISP (defined in infinite_isp.py) object to generate a .png output file. All you have to provide is the raw file and a parameter configuration (config file). A config file is used to customize pipeline behavior by setting preferences and to separate configuration details from code, enhancing flexibility and maintainability of the project. For convenience, some sample raw files are also provided in the ./in_frames directory and a default config file can be found in the ./config directory. Refer to the upcoming tutorials for details on the remaining folders in the main directory.

Configuring Script Parameters

Let’s start with processing a single raw file using the default setting of the infinite-ISP pipeline. Open the isp_pipeline.py file and set the three path parameters.

Isp_pipeline.py script

Here we will use the ColorChecker_2592x1536_10bit_GRBG raw file from the ./in_frame/normal directory and the default config file present in the ./config directory. The filename is an optional parameter in this file. Providing the filename here or in the config file under the platform heading is your choice. As the filename is set in the config file, we are good to go! 

Default config file

Use the following command on the terminal or run the isp_pipeline file in your code editor:

python isp_pipeline.py

Executing the pipeline generates a .png output file that is placed in the ./out_frames folder along with a copy of the config file.

out_frames directory

Pipeline output

Logs are also printed in the terminal briefly outlining the sequence of execution of the 18 ISP modules, their parameter configuration and the time taken to process the image as shown in the image below.

Terminal Logs

The detailed description of the modules algorithms and parameters can be found in the docs directory and README.md file.

Another raw file can be used to run the pipeline using the same configuration by setting the filename parameter in the isp_pipeline.py file.

Changing the filename parameter only 

Saving Modular output

The is_save flag  for the respective modules in the config file can also be used to generate modular level outputs. 

Is_save flag is defined for each module

These results can be saved as .npy or .png files. The save format can be configured in the config file via the “save_format” parameter under the platform section. The image shown below, for example, is the result of the demosaic module. The module level results can be accessed in the ./test_vector/Results directory, which are created on run time.

Output of the demosaic module with all previous modules enabled.

In order to use your own raw data, update the datapath variable and default config file accordingly.