Infrared(IR) small dim target detection plays an important role in precise guidance^[1], early warning^[2], and maritime target searching^[3], etc. Especially, as targets usually move very fast in real applications, real time detection within a single frame is much more important^[4]. However, it is usually a difficult task to achieve a real time IR small dim target detection with a high detection rate and a low false alarm rate within a single frame because of the following facts: (a) in the obtained IR images, due to the long distance between the target and the IR detector, the target usually occupies only a few of pixels with a dim gray level, and no shape or texture information can be utilized, which results in a low detection rate^[5], (b) some background regions may have higher brightness than real target, which results in a high false alarm rate^[6], (c) complex background edges may be falsely detected as targets, which results in a high false alarm rate, too^[7], and (d) Pixel-sized Noises with High Brightness (PNHB) may cause serious interferences, which will further reduce the detection performance^[8].

A number of single frame based algorithms for detecting IR small dim targets have been proposed, including spatial domain algorithms^[9-10], frequency domain algorithms^[11-12], morphological algorithms^[13], background estimation algorithms^[14-15], etc. Learning based algorithms, including supervised types^[16-17] and unsupervised types^[18-19] which reply on the prior knowledge or not, have been also studied. However, the detection of IR small dim targets under complex backgrounds is still an open issue.

In recent years, robust Human Visual System (HVS) properties have been introduced to the IR small dim target detection field. According to the contrast mechanism of HVS, it is the contrast but not the brightness that occupies the most important part in the streams of our visual system^[20]. This fact has a significant value for IR small dim target detection, because even the small target is not the brightest part in the whole image (i.e., high brightness background regions exist), it is usually brighter than its immediate neighborhood region and has a small local contrast. Thus, the single frame detecting algorithms based on HVS can usually achieve a better performance than traditional algorithms. However, for an algorithm based on HVS, how to define the formula for local contrast is one of the key issues, which directly determines the performance of the algorithm.

By now, researchers have not reached a consensus on how to define the local contrast, and many local contrast definitions have been proposed. Generally, existing algorithms can be divided into two categories: the difference form local contrast and the ratio form local contrast^[21]. But unfortunately, none of the existing definitions can achieve the goal of enhancing real targets while suppressing all the interferences, including high brightness backgrounds, complex background edges, and PNHB.

The difference form local contrast algorithms, such as the Laplacian of Gaussian (LoG) filter^[22-23], the Difference of Gaussian (DoG) filter^[24-26], the Improved Difference of Gabor (IDoGb) filter^[27], the Accumulated Center-Surround Difference Measure (ACSDM)^[28], and the Multiscale Patch-based Contrast Measure (MPCM)^[29], etc., calculate the difference between the center area and the surrounding area, so that high brightness backgrounds can be eliminated. Besides, some of them (such as IDoGb, ACSDM, and MPCM) take into account the directional information to suppress complex background edges better, since true small dim target is usually assumed as rough circles without anisotropy and prevailing directions, while background edges usually distribute along a particular direction in a local small area. However, these algorithms cannot enhance small dim targets effectively.

The ratio form local contrast algorithms, such as the Local Contrast Measure (LCM)^[30], the Improved Local Contrast Measure (ILCM)^[8], the Novel Local Contrast Measure (NLCM)^[31], and the Weighted Local Difference Measure (WLDM)^[32-33] etc., take the ratio between the center area and the surrounding area as an enhancement factor of the center, and could effectively enhance the small dim targets. Besides, directional information is adopted in most of them to suppress complex background edges better. However, they use the absolute local contrast and cannot effectively eliminate high brightness backgrounds.

In theory, the difference form local contrast and the ratio form local contrast have their respective advantages and disadvantages: the difference form local contrast can effectively eliminate high brightness backgrounds, but cannot effectively enhance real target; the ratio form local contrast can effectively enhance real target, but cannot effectively eliminate high brightness background. It will be expected to combine the advantages of the two if they can be used together. The simplest way is to use them as two separate stages. For example, the ILCM and the NLCM are both ratio form local contrast algorithms, but they utilized the DoG filter as a preprocessing algorithm to eliminate high brightness backgrounds first. However, dividing an algorithm into two stages may damage the robustness of the algorithm, because any error in any stage will affect the final detection performance. Other two-stage algorithms, such as LCM + LoG^[34], LCM + Support Vector Machine (SVM)^[35-36], and LCM + Local Self Similar (LSS)^[37], etc., also have the similar defections. Besides, existing local contrast algorithms (including ratio form and difference form) just simply taken the local surrounding areas as background but did not taken into account the diversity of the local surrounding background itself, which means the important information that can be used to further suppress complex backgrounds is wasted.

In this paper, a Multi-scale Ratio-Difference joint Local Contrast Measure (MRDLCM) is proposed. It integrates the ratio form local contrast and the difference form local contrast as a whole, and can combine the advantages of the ratio form methods and the difference form methods so no preprocessing algorithm is needed. After the MRDLCM calculation, different sizes of small dim targets can be enhanced by utilizing the ratio form local contrast; high brightness backgrounds can be eliminated by utilizing the difference form local contrast; complex background edges can be suppressed by introducing the directional information; and PNHB can be suppressed by the reasonable design of the formula. Besides, a weighted function utilizing the Reversed Local Diversity (RLD) of the local surrounding areas is proposed to suppress the complex backgrounds further.

The contributions of this paper can be summarized as follows: (1) A new local contrast measure named MRDLCM is proposed, it can effectively enhance IR small dim targets while suppressing high brightness backgrounds, complex background edges and PNHB simultaneously. (2) A weighted function named RLD is proposed, it utilizes the local diversity of the local surrounding areas to suppress the complex backgrounds further. (3) An IR small dim target detection algorithm based on MRDLCM_RLD is proposed, it doesn’t need any preprocessing algorithm (such as DoG or LoG, etc.) and has a simple structure, so its robustness can be guaranteed.

Real IR sequences are used to test the performance of the proposed algorithm versus some existing state-of-the-art algorithms. Experimental results demonstrate that the proposed algorithm can successfully detect the small dim targets under complex backgrounds and can achieve the best detection performance against existing algorithms in detection rate and false alarm rate. Besides, the proposed algorithm has the potential of parallel processing, which is very useful for improving the detection speed.

This paper is organized as follows. In Section I, the calculations of MRDLCM and RLD are described in details, and the detection ability is analyzed; in Section II, the detection of the small dim target based on MRDLCM_RLD is described; experimental results are presented in Section III, and this paper concludes in Section IV.

In this section, firstly the different features between real IR small dim target and other interferences are analyzed, including high brightness background, complex background edges, and PNHB. Based on the analysis, a new local contrast method named MRDLCM to enhance real target while suppressing all the interferences is proposed, and a weighted function named RLD to suppress the complex backgrounds further. The detection ability of the proposed algorithm is analyzed in this section too.

A sample of real IR image which contains a real small dim target^[38] is given in Fig.1(a). The 3D grayscale distributions (after normalization) of different types of components are shown in Fig.1(b), including true small dim target (denoted by TT), normal background (denoted by NB), high brightness background (denoted by HB), complex background edge (denoted by EB), and PNHB (denoted by PNHB).

Figure 1.  (a) A sample of real IR image; (b) 3D distributions of different types of components. Here, TT represents true small target, NB represents normal background, HB represents high brightness background, EB represents complex background edge, and PNHB represents Pixel-sized Noises with High Brightness

From Fig.1 we can get the conclusions as follows:

(1) True small dim target usually concentrates in a small, compact area that attenuates from the center, it usually emerges in flat, homogenous background zones, and is usually brighter than its immediate neighboring background in the spatial domain since the target is usually hotter. In other words, it has a small local contrast. Besides, true small dim target is usually assumed as rough circles without anisotropy and prevailing directions^[32].

(2) Normal background is usually dark and flat, it has correlation in the spatial domain and is not salient both in gray value and in local contrast.

(3) High brightness background may have a gray value far larger than real small dim target, but it usually has a large area and has correlation in the spatial domain too, thus, its local contrast is not obvious.

(4) Complex background edge also has local contrast information between the two sides, but background edge usually distributes along a particular direction in a local small area, which is different from true small dim target.

(5) PNHB has the most similar pattern to true small dim target, however, a PNHB which is usually caused by random electrical noises only emerges as a single pixel, but a true small dim target usually has a small area due to the optics Point Spread Function (PSF) of the thermal imaging system at a long distance^[36].

MRDLCM describes a pixel position by generating a signal value. As mentioned before, a small dim target is salient in a local area but not in the whole image, so we will focus on a local small image patch (as shown in Fig.2), and define the MRDLCM of the central pixel in the central cell cell(0) utilizing the ratio and the difference between cell(0) and cell(i) (i=1,2,···,8 represents different directions). See below for details. Here the central cell is used to catch real target and the surrounding cells are used to capture the surrounding backgrounds, so the cell size N should be close to or slightly larger than a real target. If the target size is unknown, to ensure that a cell can contain the total target while introducing as few interferences as possible, N should be approximated to the general maximal size of small targets^[31]. According to the Society of Photo-Optical Instrumentation Engineers (SPIE), a small target is usually smaller than 9×9^[39], so N is suggested to be set to about 9×9.

Figure 2.  A local small image patch used for MRDLCM calculation. It is divided into 9 cells, and the cell size N should be close to or slightly larger than real target

(1) The Ratio Form Local Contrast Measure (RFLCM)

First, considering real small dim target is usually the brightest in a local area, and normal background is usually dark and flat, to enhance the target, a new ratio form local contrast measure is proposed, and the RFLCM of the central pixel in cell(0) for the ith direction is defined as

where Imean₀ denotes the average gray of the K max pixels in cell(0), Imean_i denotes the average gray of the K max pixels in cell(i), shown in Eqs.(2) and (3):

where K is the number of maximal gray values considered, G^j₀ and G^j_i are the jth maximal gray value of cell(0) and cell(i). The average operation here is used to reduce the interference caused by PNHB, so K is suggested to be set to a value larger than 1. Besides, considering true target usually attenuates from its center, to get a larger enhancement on real target, K is suggested to be set to a value smaller than the target size.

Figure 3 illustrates the different cases when the central pixel of cell(0) is different. Similar to Fig.1, TT means the central pixel of cell(0) is a true target pixel, NB means the central pixel of cell(0) is a normal background pixel, HB means the central pixel of cell(0) is a high brightness background pixel, EB means the central pixel of cell(0) is a complex background edge pixel, and PNHB means the central pixel of cell(0) is a PNHB.

Figure 3.  Different cases when the central pixel of cell(0) is different

From Fig.3, it can be easily deduced that for TT and NB there will be

Since normal background usually occupies a large area and real target is usually salient in local. Therefore, true small dim target can be enhanced using the proposed RFLCM.

For PNHB, even if its gray value is close to or slightly larger than a real target, the average operation in (2) will suppress PNHB effectively if K is larger than 1, so it can be easily deduced that

At last, to suppress complex background edges, the direction information is utilized and the RFLCM is finally defined as

It can be easily deduced that

Therefore, both PNHB and complex background edges can be suppressed using the proposed RFLCM.

(2) The Difference Form Local Contrast Measure (DFLCM)

Then, considering real small dim target is usually local salient, and high brightness background usually has a large area and has correlation in the spatial domain, a new difference form local contrast measure is proposed to eliminate the high brightness background, and the DFLCM of the central pixel in cell(0) for the ith direction is defined as

where Imean₀ and Imean_i are same as Eqs.(2) and (3).

From Fig.3, it can be easily deduced that for TT and HB there will be

Since high brightness background usually has a large area and has correlation in the spatial domain, while real target is usually local salient. Therefore, high brightness background can be eliminated using the proposed DFLCM.

Similar to RFLCM, the final DFLCM is defined as

and the conclusions in (6) and (8) are easily to be proved as true for DFLCM, too, see Eqs.(13) and (14), i.e., both PNHB and complex background edges can be suppressed using the proposed DFLCM.

(3) The Ratio-Difference joint Local Contrast Measure (RDLCM)

To combine the advantages of the ratio form local contrast and the difference form local contrast, the Ratio-Difference joint Local Contrast Measure (RDLCM) is proposed here, and the method to calculate the RDLCM for a raw IR image is shown in the Algorithm 1.

Algorithm 1 RDLCM calculation.

Input: A raw IR image, the cell size N, and the parameter K.

Output: The RDLCM calculation result matrix RDLCM.

1: Form 9 cells as a patch, as shown in Fig.2.

2: Slide the patch on the raw IR image from left to right and top to down.

3: At each pixel, calculate its RFLCM and DFLCM according to Eqs.(1), (7) and Eqs.(9), (12).

4: After the calculation is end for the total image, form the results as two new matrixes RFLCM and DFLCM respectively.

5: Normalize the elements in RFLCM to the range (0, 1).

6: Normalize the elements in DFLCM to the range (0, 1).

7: Calculate the RDLCM of the raw IR image using the Hadamard product of RFLCM and DFLCM:

(4) The MRDLCM

The K in Eqs.(2) and (3) is a key parameter in the proposed algorithm. To get a better detection performance, K should be adjusted adaptively with the size of the target. However, in real applications, target size is usually unknown. Thus, the multi-scale detection is utilized in the proposed algorithm, and the calculation of MRDLCM for a raw IR image is shown the Algorithm 2 when L is the number of scales:

Algorithm 2 MRDLCM calculation.

Input: A raw IR image, the cell size N, and the parameters K₁, K₂, ···, K_L for L scales.

Output: The MRDLCM calculation result matrix MRDLCM.

1: for s=1,2,···,L do

Calculate the RDLCM_s according to the Algorithm 1 using K_s.

2: end for

3: At each pixel, output the maximum RDLCM value of the L scales as the final MRDLCM value, i.e.,

where (i, j) is the coordinate of each pixel.

Existing local contrast algorithms (including ratio form and difference form) just simply taken the local surrounding areas (cell(1)-cell(8) in Fig.2) as background when they calculate the local contrast information, but did not taken into account the diversity of the local surrounding background itself. In fact, complex backgrounds are the most common interferences in IR small dim target detection, and it’s obviously that the more complex the background, the larger the local diversity, while a real target which emerges in flat, homogenous background zones will has a smaller local diversity. Thus, the local diversity information can be used to suppress complex backgrounds further.

In this paper, two aspects are considered in the Local Diversity (LD): the diversity within each cell, and the diversity between cell(1)-cell(8). First, the diversity within each cell is defined as

where Imax_i and Imin_i are maximum and minimum gray value in cell(i) respectively. It can be easily deduced that if a cell is made up of a flat, homogenous region, the diversity within the cell will be close to 0; on the other hand, if a cell is made up of complex background, the diversity within the cell will be far larger than 0.

Then, the LD of the central pixel of cell(0) will be defined as

In fact, Eq.(18) is the variance of Din_i, i.e., it takes into account the diversity between cell(1)-cell(8). It can be easily deduced that if a pixel is located at a flat, homogenous region, its LD will be close to 0; on the other hand, if a pixel is located at complex background, its LD will be far larger than 0.

Based on the analysis above, it can be seen that for a raw IR image, a target pixel will have a smaller LD, and a complex background pixel will have a larger LD. Then, we can use the LD as a weight function to further suppress complex background. However, to ensure that a target pixel will have a larger weight value and a complex background pixel will have a smaller weight value, the normalize and reverse operation will be needed and the RLD is proposed. The calculation of RLD is shown in the Algorithm 3.

Algorithm 3 RLD calculation.

Input: A raw IR image, the cell size N.

Output: The weight matrix RLD of the image.

1: Form 9 cells as a patch, as shown in Fig.2.

2: Slide the patch on the raw IR image from left to right and top to down.

3: At each pixel, calculate its LD according to Eqs.(17) and (18).

4: After the calculation is end for the total image, form the results as a new matrixes LD.

5: Normalize the elements in LD to the range (0, 1).

6: Reverse:

For a raw IR image, after the MRDLCM and the RLD are calculated, the Multi-scale Ratio-Difference Local Contrast Measure weighted by Reversed Local Diversity (MRDLCM_RLD) will be calculated according to the Algorithm 4.

Algorithm 4 MRDLCM_RLD calculation.

Input: The matrix MRDLCM, and the matrix RLD.

Output: The MRDLCM_RLD calculation result matrix MRDLCM_RLD.

1: Calculate the MRDLCM_RLD of the raw IR image using the Hadamard product of MRDLCM and RLD:

Then, the MRDLCM_RLD results for different types of pixels (see Fig.3) will be discussed.

(1) For a TT, since a true target usually concentrates in a small, compact area and is hotter than its neighboring background, its RFLCM (before normalization, the same below) will be larger than 1, DFLCM (before normalization, the same below) will be larger than 0, then its MRDLCM will be large. Besides, since a true target usually emerges in flat, homogenous background zones, its RLD will be large, too. Thus, there will be

(2) For a NB, since normal background is usually dark and flat, its RFLCM will be close to 1, and its DFLCM will be close to 0, then its MRDLCM will be smaller than a TT’s. Thus, there will be

although the RLD of a NB may be close to the RLD of a TT.

(3) For an HB, similar to NB, it can be easily deduced that

(4) For an EB, since background edge usually distributes along a particular direction in a local small area, the minimum operation in Eqs.(7) and (12) ensures that the EB’s RFLCM will be close to or smaller than 1, DFLCM will be close to or smaller than 0, then its MRDLCM will smaller than a TT’s. Besides, the RLD of an EB will be smaller than a TT’s, too. Thus, there will be

(5) For a PNHB, since PNHB usually emerges as a single pixel and the average operation is utilized in Eqs.(2) and (3), the PNHB’s RFLCM and DFLCM will be smaller than a TT’s if K is set to a value larger than 1, even the PNHB has a same gray value with TT. Thus, there will be

although the RLD of a PNHB may be close to the RLD of a TT.

From the discussions above it can be seen that after the MRDLCM_RLD calculation, true small dim target will be the most salient, while all the other interferences are suppressed, which means the proposed MRDLCM_RLD has a good detection ability for IR small dim target, and no preprocessing is needed.

The flow chart of the whole detection algorithm is shown in Fig.4. The MRDLCM and RLD will be first calculated for a raw IR image respectively, then the RLD result will be used as a weight function for MRDLCM, and the MRDLCM_RLD is calculated. From Fig.4 it can be seen that after the MRDLCM_RLD calculation, true small dim target will be the most salient in the saliency map MRDLCM_RLD, so a threshold operation can be used to extract target. Besides, Fig.4 also hints that the proposed algorithm has the potential of parallel processing.

Figure 4.  Flow chart of the proposed algorithm

There are many types of threshold definitions. In this paper, we adopt the idea of the widely used Gaussian threshold because it is an adaptive threshold definition and can be effectively used to capture the abnormal salient information in a large number of data, and the threshold Th is adaptively defined as

where μ and σ are the mean and standard deviation of MRDLCM_RLD, k_th is a given parameter. Our experiments show that the optimal range of k_th is from 2 to 7.

In MRDLCM_RLD, the pixels which have larger value than Th will be output as target pixels, while other parts are discarded. In the final detection result, each connected area will be regarded as a detected target (In order to eliminate clutters, a dilation operation may be needed).

Detection speed is an important index in the field of IR small dim target detection since real time detection is usually needed in many practical applications. Parallel processing is a common method to improve the detection speed as parallel processing devices such as GPU have been widely used. In fact, in most cases the lack of parallel processing capability is the key problem that affects the detection speed, so it is necessary to analyze the parallel processing capability of the proposed algorithm.

It can be easily deduced that the proposed algorithm has the potential of parallel processing. First, the calculations of MRDLCM and RLD can be carried out in parallel; then, the calculations of different scales can be carried out in parallel. See Fig.4. Besides, for each scale, the calculations of each pixel can be carried out in parallel; for each pixel, the calculations of each direction can be carried out in parallel.

In this section, four real IR sequences are used to verify the performance of the proposed algorithm. First, the features of different sequences are introduced. Then, the key parameter K in the proposed algorithm is optimized using different sizes of targets. Then, the detection results for different sequences of the proposed algorithm are given. To further illustrate the effectiveness of the proposed algorithm, the comparisons of the detection performances between the proposed algorithm and other state-of-the-art algorithms are given, too. All the experiments are conducted on a computer with 4 GB random access memory and 3.4 GHz Intel i3 processor, and the code was implemented in MATLAB R2016b.

In this paper, four real IR sequences which contains different sizes of small dim targets and different types of backgrounds are used to verify the performance of the proposed algorithm. The samples for each sequence are shown in Fig.5.

Figure 5.  Samples for the four IR sequences. (a) A sample for Seq. 1; (b) A sample for Seq. 2; (c) A sample for Seq. 3; (d) A sample for Seq. 4

From Fig.5 it can be seen that the targets are very small and dim, and the backgrounds are complex. The details of the features of different sequences are given in Table 1.

Besides, Table 2 gives some characteristics of the first frame of the four sequences, here C_wh, C_nb, and SCR are defined as follows, and I_t is the maximum gray value of the target, I_wh is the maximum gray value of the whole image, I_nb is the mean gray value in the neighborhood area of the target, σ_wh is the stand deviation of the whole image:

From Fig.5 and Table 2 we can see that for the four sequences been used in this paper, the targets may be not the brightest part in the whole image, i.e., the C_wh of the four sequences are all smaller than 1. For example, the C_wh of Seq. 1 or Seq. 3 is just about 0.3. However, in a local region, the targets are all brighter than their neighborhood areas, i.e., the C_nb of the four sequences are all larger than 1. Even in the worst case in Seq. 2 or Seq. 3, the C_nb is still larger than 1.16. Besides, complex backgrounds and heavy clutters bring a low SCR, especially for Seq. 3, which has a SCR lower than 0.2.

In the proposed algorithm, the K in Eqs.(2) and (3) for MRDLCM calculation is a key parameter. It is obviously that to reduce the interference caused by PNHB, K should be larger than 1; but, to get a better enhancement on real target, K should be adjusted adaptively with the size of the target. Considering target size is usually unknown in practice, multi-scale detection is adopted in this paper (see the Algorithm 2), then the optimization of the parameters K₁, K₂, ···, K_L (L is the scale number totally used) for each scale will be an essential matter.

In this section, numerous simulations have been done to choose the optimal K for different target sizes. The 2D Gaussian function is used to model the small target, and the maximum gray value of the target is set to 120. Different target sizes including 3 × 3, 5 × 5, 7 × 7 and 9 × 9 are tested. The resolution of the simulated images is set to 320 × 256. The gray vale of the normal background is set to 100, particularly, in the left-up corner there is a small area (80 × 240) of high brightness background with gray value 200. Target is located at (156, 180) in the first frame, and moves from left to right one pixel per frame. Random noises with a standard variation of 20 are added to each simulated image.

Figure 6 gives the SCR value using different K for different target sizes, for each target size 20 frames are listed here. From Fig.6 it can be seen that for a smaller target, K=2 will be proper; for a larger target, K=4 will be proper.

Figure 6.  SCR results before and after MRDLCM calculation using different K for simulated data. (a) Target size is 3 × 3; (b) Target size is 5 × 5; (c) Target size is 7 × 7; (d) Target size is 9 × 9

To verify this conclusion, three real sequences including Seq. 1, Seq. 2 and Seq. 3 are used for test, too. The results are shown in Fig.7, for each sequence 10 frames are listed here. It can be seen that in most cases the optimal value of K is 2 or 4, too. Thus, without loss of generality, 2 scales are used in this paper (i.e., L=2) and K₁ is set to 2, K₂ is set to 4.

Figure 7.  SCR results before and after MRDLCM calculation using different K for real sequences. (a) Seq. 1, target size is 7 × 5; (b) Seq. 2, target size is 5 × 5; (c) Seq. 3, target size is 3 × 3

First, to verify the effectiveness of the proposed algorithm, the calculation results for different types of pixels are given in Fig.8. The same figure with Fig.1 and Fig.3 is used here.

Figure 8.  Calculation results for different types of pixels using the proposed MRDLCM_RLD algorithm. (a) Different cases when the central pixel of cell(0) is different; (b) The calculation result MRDLCM_RLD for the whole image using the proposed MRDLCM_RLD algorithm; (c) The 3D distributions of different types of components

Comparing Fig.8(c) to Fig.1(b) it can be seen that real small dim target is enhanced using the proposed algorithm, while other interferences all been suppressed, which is consistent with the discussions in Section 1.4.

Figure 9 gives the detection results of different sequences using the proposed MRDLCM_RLD algorithm. Here K₁ is set to 2, K₂ is set to 4, and N is 9 × 9. The same samples with Fig.5 are used here.

Figure 9.  From top to bottom: the detection results using the proposed MRDLCM_RLD algorithm for Seq. 1, Seq. 2, Seq. 3 and Seq. 4. (a) The raw IR image samples of the four sequences; (b) The DFLCM result; (c)The RFLCM result; (d)The MRDLCM result; (e) The RLD result; (f) The MRDLCM_RLD result; (g) The threshold operation results, each connected area is regarded as a target

It can be seen from Fig.9 that in the raw IR images, the targets are small and dim, and complex backgrounds and heavy clutters exist, see Fig.9(a). After the DFLCM calculation, the complex background can be suppressed to some extent, but the target is still weak, especially when the target is small and dim in the raw images, see Fig.9(b). In the RFLCM calculation results, the target is usually larger, see Fig.9(c), thus, in the results after MRDLCM calculation, the targets can be enhanced, while complex backgrounds and heavy clutters are suppressed further, see Fig.9(d). After the RLD calculation, the homogenous background area will get a large value, and the complex background area will get a small value, see Fig.9(e). Thus, when we weighted MRDLCM with RLD, the complex backgrounds and heavy clutters will be suppressed further, see Fig.9(f). At last, real small dim targets are output correctly for both single target cases and multi targets cases by the threshold operation, while other parts are discarded, see Fig.9(g). Only in Seq. 3 there is a false alarm since Seq. 3 has a very small dim target and very complex backgrounds (see Table 2).

To better explain the effectiveness of RLD, Fig.10 gives the detection result using only MRDLCM alone for Seq. 3. It can be seen that there will be more false alarms when RLD is removed, which proves from the opposite side that RLD can further suppress complex backgrounds.

Figure 10.  Detection result using only MRDLCM alone for Seq. 3. (a) The raw IR image sample of Seq. 3; (b) The MRDLCM result; (c) The threshold operation result on MRDLCM, more false alarms emerge

To further illustrate the effectiveness of the proposed algorithm, six state-of-the-art algorithms are chosen for comparison, including DoG^[24], MPCM^[29], ILCM^[8], NLCM^[31], WLDM^[32], and RLCM^[21]. DoG is a traditional difference form local contrast method; MPCM is a multi-scale patch based difference form local contrast method; ILCM and NLCM are ratio form local contrast method, and they both utilize DoG as preprocessing stage; WLDM utilizes the local entropy as a weight function for local contrast; RLCM take the difference between the enhanced cell(0) and the original cell(0) as local contrast, it is a newly proposed multi-scale local contrast method in which both ratio operation and difference operation are used.

First, the single frame detection results using the comparison algorithms are shown in Fig.11, the same samples with Fig.9 are used here.

Figure 11.  Comparisons of detection results between different algorithms, from top to down: the detection results of Seq. 1, Seq. 2, Seq. 3 and Seq. 4 using (a) DoG; (b) ILCM; (c) NLCM; (d) WLDM; (e) MPCM; and (f) RLCM

From Fig.11 it can be seen that:

(1) For Seq. 1, DoG, ILCM, NLCM and WLDM cannot detect the small target correctly but a lot of false alarms emerge; MPCM can detect the small target, but false alarm emerges too; only RLCM can detect the small target correctly without any false alarms.

(2) The case of Seq. 2 is similar to that of Seq. 1.

(3) For Seq. 3, even MPCM cannot detect the target successfully since the target is too dim and the backgrounds are too complex. RLCM, on the other hand, although can detect the small target, presents more false alarms than the proposed algorithm.

(4) For Seq. 4 which contains multi targets, since the targets are somewhat bright and the backgrounds are homogenous, almost all the algorithms can successfully detect the four targets without any false alarms, except DoG, which detects only three real targets but presents two false alarms.

Comparing Fig.11 to Fig.9, it is easy to get the conclusion that the proposed algorithm can achieve the best detection performance against other existing algorithms.

Then, the comparison results for the whole sequences using different algorithms are also given in Fig.12, here the Receiver Operating Characteristic (ROC) curves^[40] are used and the False Positive Rate (FPR) and the True Positive Rate (TPR) are defined as (30) and (31).

Figure 12.  ROC curves of different algorithms for (a) Seq. 1, (b) Seq. 2, (c) Seq. 3 and (d) Seq. 4

From Fig.11 and Fig.12 it can be seen that:

(1) The detection performance of DoG is the worst, because DoG is a difference form local contrast algorithm, it cannot effectively enhance the small dim target. Besides, it doesnot utilize the directional information and cannot distinguish complex background edges.

(2) WLDM has a better performance than DoG, because WLDM is a ratio form local contrast method and can enhance the small dim targets effectively. However, it can’t eliminate high brightness backgrounds effectively.

(3) ILCM has a better performance than DoG and WLCM, because it is a ratio form local contrast method and takes DoG as preprocessing, so it can enhance small dim targets and suppress high brightness backgrounds simultaneously. Besides, the directional information is utilized in ILCM to better suppress the complex background edges.

(4) The case of NLCM is similar to ILCM, and their performances are similar too.

(5) MPCM adopts multi-scale detection can achieve a satisfied detection performance in Seq. 1, Seq. 2, and Seq. 4, but in Seq. 3, its performance is the worst, because the target is too small and too dim, and the backgrounds are too complex in Seq. 3. In other words, the robustness of MPCM is not good. The reason is that MPCM is a difference form local contrast algorithm, it cannot effectively enhance the small dim target.

(6) As a newly proposed multi-scale local contrast method, RLCM can achieve a better detection performance than other existing algorithms in all sequences since it utilizes both ratio operation and difference operation, but its performance in Seq. 3 is still not good.

(7) The proposed algorithm, which joints the ratio form local contrast and the difference form local contrast together to enhance real target and suppress interferences simultaneously, can achieve the best detection performance with good robustness in all the four sequences. Especially, for Seq. 3 in which real target is dim and backgrounds are complex, the detection performance of the proposed algorithm is greatly improved comparing to RLCM.

In this paper, a Multi-scale Ratio-Difference Local Contrast Measure (MRDLCM) weighted by Reversed Local Diversity (RLD) for IR small dim target detection is proposed. MRDLCM can combine the advantages of the ratio form methods and the difference form methods, so it can suppress all the types of interferences while enhancing different sizes of real targets, and do not need any preprocessing algorithms. RLD utilizes the local diversity information to suppress the complex backgrounds further. Four real IR sequences which contain different types of backgrounds and different sizes of targets are used for experiments, and the experimental results show the effectiveness and the robustness of the proposed MRDLCM_RLD algorithm in detection rate and false alarm rate against six existing state-of-the-art algorithms. Besides, the proposed algorithm has the potential of parallel processing, which is very useful for improving the detection speed.