Chips&Media is partnering with BTREE to provide packaged multimedia IPs, including ISP IPs, to offer the best solution possible to the clients with a complete pipeline with optimized gate size and memory usage.

Multi-exposure High Dynamic Range (HDR) CP

As human eyes have a wider dynamic range than CMOS image sensors (CIS), a wider range of brightness can be recognized. A scene or an object which is visible to human eyes is not expressed clearly (either having oversaturation or under-saturation). To compensate for this factor, a CIS and IP algorithm to observe a higher range of brightness was developed.

The result of High Dynamic Range (HDR) based on single exposure (1 image) has a limitation of not being able to regenerate the saturated region from the CIS. To resolve this limitation, an algorithm that can synthesize multiple exposures (by combining more than 1 image) is being utilized. This new algorithm is called Wide Dynamic Range (WDR), which is also referred to as multi-exposure HDR.

The NIX IP analyzes long exposed images and short exposure images from the CIS, and performs fusion, local and global tone mapping, anti-ghosting, and contrast correction, then outputs the image with a wide dynamic range of 108 dB or higher.

For real-time processing, NIX provides a Video Stream Interface, which is similar to the output format from the CIS, and an AXI interface can also be provided per the customer’s requirements.

​

Performance

• Max. resolution:  8MP (3840h x 2160v)​

• Max. input frame rate:  60fps (@ 8MP, Output : 30 fps)

​

Features

• Low power consumption and small gate count

• 20-bit Sensor Data Interface

• Anti-Ghosting

• Adaptive Tone Mapping per input images

• Real-time HDR Output

8M, 3 Exposure HDR

The CMOS Image Sensor (CIS) and IP algorithms have been developed to observe a broader range of brightness similar to the human eye. Wide Dynamic Range (WDR), which is based on 'single exposure (1-image)', has the limitation of not regenerating the 'saturation region' in CIS. A new algorithm, High Dynamic Range (HDR), also called Multi-Exposure HDR, for synthesizing 'multi-exposure (combining more than one image)' is used to resolve the limitation. 

​

STYX is capable of creating a radiance map (also called Fusion) that uses long, mild, and short exposed images from CIS. The anti-ghosting algorithm minimizes the ghost effects that occur based on the time difference between the long and short images. After the local and global tone-mapping performance has performed, it produces of WDR image of 120dB or more. STYX provides a video stream interface, similar to the CIS output format and AXI interface per customer's requirements for real-time processing. 

​

Performance

• Max. resolution:  

  • Image: 8MP​

  • Video: 8MP 

• Input formats (Bayer): 

  • HDR Linear Mode: Max 24 bits​

  • Companding mode: 12, 14, 16 bits 

  • Interleaved Mode (long, mild, short): 14 bits

• Output formats (Bayer):

  • Bayer: 14 bits​

​​

Features

• Long/mild/short image fusion

• Anti-ghosting

• Tone mapping

• Contrast correction

• Real-time HDR output

• Low power consumption 

• Small gate count

• 24 bit sensor data interface

Size optimized, competitive quality ISP

The Camera ISP IP – CARPO is the Imaging Signal Processing (ISP) targeted to be used in low light environments for surveillance cameras, automotive applications such as car DVRs, and Infotainment systems with a maximum processing resolution of 2 Mega Pixels (MP) at 30 frames per second (FPS). The ISP offers a complete and configurable ISP pipeline with features including 8x6 2D Color-Shading Correction, 19-Point Bayer Gamma Correction, 6-Region Color Saturation, Hue, and Delta-L Control functions, and generates the best possible picture quality by implementing optimum low light Noise/Sharp filters on the inputted image after analyzing each scene, which is one of the key advantages of CARPO ISPs. Furthermore, the other competitive advantages over other IPs are the significantly low gate size and memory usage which are achieved by optimizing the algorithm through information sharing of each process. ​

​

Performance

• Max. resolution:  Full-HD (16:9) 1920(H) x 1080(V)​

• Max. frame rate:  30fps@ 2MP

​

Features

• Defective Pixel Correction​

• Gb/Gr Unbalance Correction

• 2D Lens-Shading Correction (8x6)

• Supports Color Stain Correction

• High-Resolution RGB Interpolation

• Color Correction Matrix (3x3)

• Bayer Gamma Correction (19 points)

• Color Enhancement (Hue /Sat /∆-L)

• High-Performance Noise Reduction for low light environment

• High-Resolution Sharpness Control

• Auto Exposure

• Auto White Balance

• Tone Mapping

• Auto Focus

 

Interface

• ARM® AMBA 4 APB bus interface for CARPO ISP system control

• 128-bit ARM® AMBA 4 AXI Interface for data

• Direct connection to sensor stream data

Size optimized, competitive quality ISP

The Camera ISP IP – LEDA is the Imaging Signal Processing (ISP) targeted to be used in low-light environments for surveillance cameras, automotive applications such as car DVRs and Infotainment systems and has a maximum processing resolution of 5 Mega Pixels (MP) at 30 frames per second (FPS). This ISP offers a complete and configurable ISP pipeline with features including 8x6 2D Color-Shading Correction, 19-Point Bayer Gamma Correction, 6-Region Color Saturation, Hue, and Delta-L Control functions, and generates the best possible picture quality by implementing optimum low light Noise/Sharp filters on the inputted image after analyzing each scene, which is one of the key advantages of LEDA ISPs. Furthermore, the other competitive advantage over other IPs is the significantly low gate size and memory usage which are achieved by optimizing the algorithm through information sharing of each process. ​

​

Performance

• Max. resolution: Full-HD (16:9) 2560(H) x 1920(V)​

• Max. frame rate: 30 fps@ 8MP

​

Features

• Defective Pixel Correction​

• Gb/Gr Unbalance Correction

• 2D Lens-Shading Correction (8x6)

• High-Resolution RGB Interpolation

• Color Correction Matrix (3x3)

• Bayer Gamma Correction (19 points)

• Color Correction Matrix (3x3)

• Bayer/RGB/YC Domain Noise Reduction

• High-Resolution Sharpness Control

• Uses Multi-Sharp Filter and Individual Sharp Gain Control

• Auto Exposure

• 16x16 luminance weight window & pixel weighting

• Auto White Balance

• R/G/B based feed-forward method

• Tone Mapping

• Multi-band scheme using linear transformation as multi-slope

• Auto Focus

• 2-type 6-region AF value return

​

Interface

• ARM® AMBA 4 APB bus interface for CARPO ISP system control

• 128-bit ARM® AMBA 4 AXI Interface for data

• Direct connection to sensor stream data

Quality optimized, multi-instance (multi-sensor: max 4 inputs) ISP​

The Camera ISP IP – METIS is the Image Signal Processing (ISP) IP developed to be used in automotive and surveillance applications such as Around View Monitor / Surround View Monitor, Dash Cams, and Autonomous Driving Assistants. METIS offers a complete and configurable ISP pipeline with multi-instances to process up to 4 inputs from CMOS sensors simultaneously. 

​

Performance

• Image: Max: 13M Pixel (combined simultaneous processing)​​

• Video: Max: 13M Pixel @ 60fps (combined simultaneous processing)​​

​

Formats

• Input formats: Bayer (8-,10-,12-, 14-bit)​

• output formats: YUV (4:4:4, 4:2:0, 4:2:0 (AXI only) - 8 bit)

​

Key Functions

• Defective Pixel Correction​

• 10 Bit Bayer Channel Gain Support up to x3.99

• Gb/Gr Unbalance Correction

• 2D Lens-Shading Correction (18x18)

• High-Resolution RGB Interpolation

• Color Correction Matrix (3x3)

• Bayer Gamma Correction (19 points)

• Color Enhancement (Hue / Sat / ∆-L )

• High-Performance Noise Reduction for low light environment

• High-Resolution Sharpness Control

• Auto Exposure

• Auto White Balance

• Tone Mapping

• Auto Focus

​

Interface

• ARM® AMBA APB bus interface for METIS ISP system control

• ARM® AMBA AXI Interface for data

• Direct connection to sensor stream data

Up-/down Scaler

The post-processing ISP IP, CSCALE, is applied to up-/down scale the input image to match the targeted output size. For base algorithms, it uses polynomial interpolation (PI) and other methods to obtain high-quality images based on the original size of the image. 

​

Performance

• Max input resolution: 8192x8192

• Min input resolution: 800x400 

• Output resolution: max 8192x8192 

• Scaling Ratio: 1/16 ~ 16x

• Data Type: RGB/YUV444/YUV422/YUV420 

• Bit Resolution: 8-bit

​

Features

• Sharpness Control

• Filter memory

• buffer memory

3D Noise Reduction (3DNR), CP

In the signals outputted from the CMOS Image Sensor (CIS), base noise such as Additive White Gaussian Noise (AWGN) exists. This base noise does not affect the digital image during the daytime when daylight inputted to the CIS is strong enough to prevent image deterioration. However, during night time when the available light signal is weak, the base noise level becomes equal or similar to the light signal and acts as the main cause of digital image deterioration. In still images, a mask filter-based convolution calculation is used to remove base noise. The downside of this method is that as noise increases, the mask filter size and calculations increase significantly.  In continuous stream images (i.e. video), besides using the mask filter-based method, noise is also removed by accumulating temporal noise data, which possesses Gaussian distribution that when the average input signal of unit time (temporal) is used, temporal noise can be effectively removed. However, noise reduction using sequential images (referred to as 3-dimensional noise reduction – 3DNR) generates ghosting when movement or motion appears during the image sequences due to inconsistent images of the previous and current signal. The generation of ghosting in 3DNR is inevitable and will only be decreased, therefore, in HYDRA, the anti-ghost block is included to minimize ghosting.

​

HYDRA includes Digital Video Port (DVP), which is similar to the general output format from the CIS, as a baseline interface for real-time processing. An AXI interface can also be provided per the customer’s requirements.​
 

Performance

• Maximum resolution: 8MP - 3840h x 2160v

• Maximum frame rate: 30fps

​

Features

• Low power consumption and small gate count

• Temporal noise reduction using 2 frame images

• Support anti-ghosting

• Support real-time de-noising output

13M, 3D Noise Reduction 

The signal output from the CMOS Image Sensors (CIS) has base noise such as Addictive White  Gaussian Noise (AWGN). In the daytime environment, the signal input to the CIS is strong, so digital image deterioration due to base noise does not occur. However, when the available optical signal is weak at night, the base noise level becomes similar to the optical signal, which is the primary cause of digital image deterioration. 

​

Mask Filter, based on convolution calculation, is used to remove the base noise for still images. The negativity of this filter is that the size and calculation of the mask filter increase dramatically as the noise increases. In a continuous stream of images such as video, a mask filter method and a method of removing noise by accumulating 'Temporal Noise Data' is used.  In general, the temporal noise has a 'Gaussian Distribution,' so using the average of the input signal per unit time can effectively remove temporal noise. However, 'Noise Reduction' using a sequential image (3DNR) generates the 'Ghost' phenomenon when movement or motion appears during the sequences because the previous image signal and current image signal are inconsistent. CHARON uses 'motion estimation' to suppress ghosting. 

​

VINI provides a 'digital video port (DVP) interface for the real-time process,' similar to the CIS output format. It also provides an AXI interface per customer requirements. 

​

Performance

• Maximum resolution

  • Image: 13MP

  • Video: 13 MP

• Input formats

  • YUV422 - 8 bits

• Output formats

  • DVP: YUV422 - 8 bits​

  • AXI: YUV420, YUV422 - 8 bits

​​​

Features

• Motion Estimation Method (ME)

• Temporal noise reduction using continuous accumulative frames of images

• Anti-ghost & Anti-motion blur

• 64x64 motion map

• Real-time de-noising output

• Low power consumption

• Efficient gate size and memory usage 

Lens Distortion Correction, CP

Most cameras or sensors, which use the lens to collect light, face input image distortion, or warping of straight lines (bending). This phenomenon is intensified in wide lenses, fisheye lenses, and poor quality sensors. Among this type of distortion, the most prominent one is radial distortion, in which the image expands or shrinks radially from a point on the image. There are numerous methods that can be applied to correct these distortions, with advantages and disadvantages for each. KERBEROS uses a method that estimates the parameter which determines the distortion rate through a mathematical model of the distortion, then performs a correction by inversely transforming the estimated parameter value. The KERBEROS IP is composed of 3 hardware blocks, a block generating coordinates for distortion correction, a block to store external memory to cache data using the generated coordinates, and a block that produces the final data by interpolating data stored in the data cache.​

 

Performance

• Maximum resolution: 2592x1944, 5M

• Maximum frame rate: 30fps (2592x1944)

 

Features

• Programmable window size and position

• Supports up to 100% barrel distortion correction