Abstract
Motion imagery interpretability is commonly represented by the Video National Imagery Interpretability Rating Scale (VNIIRS), which is a subjective metric based on human analysts’ visual assessment. Therefore, VNIIRS is a very time-consuming task. This paper presents the development of a fully automated motion imagery interpretability prediction, called AMIIP. AMIIP employs a three-dimensional convolutional neural network (3D-CNN) that accepts as inputs many video blocks (small image sequences) extracted from motion imagery, and outputs the label classification for each video block. The result is a histogram of the labels/categories that is then used to estimate the interpretability of the motion imagery. For each training video clip, it is labeled based on its subjectively rated VNIIRS level; thus, the required human annotation of imagery for training data is minimized. By using a collection of 76 high definition aerial video clips, three preliminary experimental results indicate that the estimation error is within 0.5 VNIIRS rating scale.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
Notes
- 1.
This is based on the standard MISB ST 0901.2. However, in the newest standard MISB ST 0901.3, criteria are defined for three orders of battle.
- 2.
The five channels are defined as gray, gradient-x, gradient-y, optflow-x and optflow-y.
- 3.
Different video block sizes are experimented in this paper.
References
MISB ST 0901.2: Video-National Interpretability Rating Scale, Feb 2014
MISB RP 1203.3: Video Interpretability and Quality Measurement and Prediction, Feb 2014
ITU-T Recommendation P.912, Subjective Video Quality Assessment Methods for Recognition Tasks, Aug 2008
Blasch, E., Kahler, B.: Application of VNIIRS for target tracking. In: Proceedings of SPIE vol. 9473 (2015)
Blasch, E., Kahler, B.: V-NIIRS fusion modeling for EO/IR systems. In: IEEE National Aerospace and Electronics Conference (2015)
Blasch, E., Chen, H-M., Wang, Z., Jia, B., et al.: Target broker compression for multi-level fusion. In: IEEE National Aerospace and Electronics Conference (2016)
Blasch, E., Chen, H-M., Wang, Z., Jia, B., et al.: Compression induced image quality degradation in terms of NIIRS. In: IEEE Applied Imagery Pattern Recognition Workshop (AIPR) (2016)
Zheng, Y., Dong, W., et al.: Qualitative and quantitative comparisons of multispectral night vision colorization techniques. Opt. Eng. 51(8), 08004 (2012)
Zheng, Y., Blasch, E., Liu, Z.: Multispectral Image Fusion and Colorization. SPIE Press (2018)
Palaniappan, K., et al.: Moving object detection for vehicle tracking in wide area motion imagery using 4D filtering. In: International Conference on Pattern Recognition (ICPR) (2016)
Snidaro, L., García, J., Llinas, J., Blasch, E. (eds.): Context-Enhanced Information Fusion. ACVPR, Springer, Cham (2016). https://doi.org/10.1007/978-3-319-28971-7
Wu, R., Liu, B., Chen, Y., et al.: A Container-based elastic cloud architecture for pseudo real-time exploitation of wide area motion imagery (WAMI) stream. J. Signal Process. Syst. 88(2), 219–231 (2017)
Al-Shakarji, N.M., Bunyak, F., Seetharaman, G., Palaniappan, K.: Robust multi-object tracking for wide area motion imagery. In: IEEE Applied Imagery Pattern Recognition Workshop (AIPR) (2018)
Aktar, R., AliAkbarpour, H., Bunyak, F., Seetharaman, G., Palaniappan, K.: Performance evaluation of feature descriptors for aerial imagery mosaicking. In: IEEE Applied Imagery Pattern Recognition (AIPR) Workshop (2018)
Zheng, Y., Chen, G., Wang, Z., et al.: Image quality (IQ) guided multispectral image compression. In: Proceedings of SPIE, vol. 9871 (2016)
Blasch, E., et al.: Prediction of compression-induced image interpretability degradation. Opt. Eng. 57(4), 043108 (2018)
Gao, K., Yao, S., AliAkbarpour, H., Agarwal, S., Seetharaman, G., Palaniappan, K.: Sensitivity of multiview 3D point cloud reconstruction to compression quality and image feature detectability. In: IEEE Applied Imagery Pattern Recognition (AIPR) Workshop (2019)
Al-Shakarji, N.M., Bunyak, F., AliAkbarpour, H., Seetharaman, G., Palaniappan, K.: Performance evaluation of semantic video compression using multi-cue object detection. In: IEEE Applied Imagery Pattern Recognition (AIPR) Workshop (2019)
Prasath, V.B.S., Pelapur, R., Seetharaman, G., Palaniappan, K.: Multiscale structure tensor for improved feature extraction and image regularization. IEEE Trans. Image Process. 28(12), 6198–6210 (2019)
Çetin, M., Stojanović, I., Önhon, N.O., Varshney, K., Samadi, S., et al.: Sparsity-driven synthetic aperture radar imaging: reconstruction, autofocusing, moving targets, and compressed sensing. IEEE Signal Process. Mag. 31(4), 27–40 (2014)
Majumder, U., Blasch, E., Garren, D.: Deep Learning for Radar and Communications Automatic Target Recognition. Artech House, Norwood (2020)
Huynh-Thu, Q., Garcia, M.N., Speranza, F., et al.: Study of rating scales for subjective quality assessment of high-definition video. IEEE Trans. Broadcast 57(1), 1–14 (2011)
Zhang, Y., Gao, X., He, L., et al.: Blind video quality assessment with weakly supervised learning and resampling strategy. IEEE Trans. Circuits Sys. Video Tech. 29(8), 2244–2255 (2018)
Li, Y., et al.: No-reference video quality assessment with 3D shearlet transform and convolutional neural networks. IEEE Trans Circuits Sys Video Tech. 26(6), 1044–1057 (2016)
Shahid, M., Rossholm, A., Lövström, B., Zepernick, H.-J.: No-reference image and video quality assessment: a classification and review of recent approaches. EURASIP J. Image Video Process. 2014(1), 1–32 (2014). https://doi.org/10.1186/1687-5281-2014-40
Vega, M.T., Sguazzo, V., Mocanu, D.C., et al.: An experimental survey of no-reference video quality assessment methods. Int. J Pervasive Comp. Comm 12(1), 66–86 (2016)
Xu, L., Lin, W., Kuo, C.C.J.: Visual quality assessment by machine learning. Springer, Berlin (2015)
Varga, D.: No-reference video quality assessment based on the temporal pooling of deep features. Neural Process. Lett. 50(3), 2595–2608 (2019)
Ji, S., Xu, W., et al.: 3D convolutional neural networks for human action recognition. IEEE Trans. Pattern Anal. Mach. Intell. 35(1), 221–231 (2012)
Karpathy, A., Toderici, G., et al.: Large-scale video classification with convolutional neural networks. In: IEEE Conference on Computer Vision and Pattern Recognition (2014)
Tran, D., Bourdev, L., et al.: Learning spatiotemporal features with 3D convolutional networks. IEEE International Conference on Computer Vision (2015)
He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 770–778 (2016)
Blasch, E., Seetharaman, G., et al.: Wide-area motion imagery (WAMI) exploitation tools for enhanced situation awareness. In: IEEE Applied Imagery Pattern Recognition Workshop (2012)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2021 Springer Nature Switzerland AG
About this paper
Cite this paper
Chen, Hm., Chen, G., Blasch, E. (2021). On the Development of a Classification Based Automated Motion Imagery Interpretability Prediction. In: Del Bimbo, A., et al. Pattern Recognition. ICPR International Workshops and Challenges. ICPR 2021. Lecture Notes in Computer Science(), vol 12668. Springer, Cham. https://doi.org/10.1007/978-3-030-68793-9_6
Download citation
DOI: https://doi.org/10.1007/978-3-030-68793-9_6
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-68792-2
Online ISBN: 978-3-030-68793-9
eBook Packages: Computer ScienceComputer Science (R0)
