Search by item HOME > Access full text > Search by item

JBE, vol. 26, no. 5, pp.489-499, September, 2021

DOI: https://doi.org/10.5909/JBE.2021.26.5.489

Analysis and Design of Arts and Culture Content Creation Tool powered by Artificial Intelligence

Choonsung Shin and Hieyong Jeong

C.A E-mail: h.jeong@jnu.ac.kr

Abstract:

This paper proposes an arts and culture content creation tool powered by artificial intelligence. With the recent advances in technologies including artificial intelligence, there are active research activities on creating art and culture contents. However, it is still difficult and cumbersome for those who are not familiar with programming and artificial intelligence. In order to deal with the content creation with new technologies, we analyze related creation tools, services and technologies that process with raw visual and audio data, generate new media contents and visualize intermediate results. We then extract key requirements for a future creation tool for creators who are not familiar with programming and artificial intelligence. We finally introduce an intuitive and integrated content creation tool for end-users. We hope that this tool will allow creators to intuitively and creatively generate new media arts and culture contents based on not only understanding given data but also adopting new technologies. 



Keyword: art and culture content, artificial intelligence, content creation tool, deep learning

Reference:
[1] Google Arts & Culture, https://artsandculture.google.com/ (accessed June. 27, 2021)
[2] The Next Rembrandt, https://thenextrembrandt.pr.co/(accessed June. 27, 2021)
[3] Magenta, https://magenta.tensorflow.org/(accessed June. 27, 2021)
[4] Deep dream generator, https://deepdreamgenerator.com/(accessed June. 27, 2021)
[5] Teachable Machine, https://teachablemachine.withgoogle.com/(accessed June. 27, 2021)
[6] D. M. Blei, A. Y. Ng, M. I. Jordan, “Latent Dirichlet Allocation,” Journal of Machine Learning Research, Vol. 3, pp. 993-1022, 2003.
[7] S. Minaee, N. Kalchbrenner, E. Cambria, N. Nikzad, M. Chenaghlu, and J. Gao, “Deep Learning-based Text Classification: A Comprehen- sive Review. ACM Computing Survey,” Vol. 54, No. 3, Article 62, 2021. doiI:https://doi.org/10.1145/3439726
[8] Z. Kastrati, L. Ahmedi, A. Kurti, F. Kadriu, D. Murtezaj, F. Gashi, “A Deep Learning Sentiment Analyser for Social Media Comments in Low-Resource Languages,” MDPI Electronics, Vol. 10, No 1133, 2021. https://doi.org/10.3390/electronics10101133
[9] tSNE Java Script demo, https://cs.stanford.edu/people/karpathy/tsnejs/ (accessed June. 27, 2021)
[10] C. Sievert, K. Shirley, “LDAvis: A method for visualizing and interpreting topics,” In Proceedings of the workshop on interactive language learning, visualization, and interfaces, pp. 63-70, 2014.
[11] A. Krizhevsky, I. Sutskever, and G. E. Hinton, “ImageNet classification with deep convolutional neural networks,” In Proc of the 25th International Conference on Neural Information Processing Systems(NIPS'12), Vol. 1,Curran Associates Inc., Red Hook, NY, USA, 1097–1105.
[12] K. He, X. Zhang, S. Ren and J. Sun, "Deep Residual Learning for Image Recognition," In Proc of IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 770-778, 2016. doi: 10.1109/ CVPR.2016.90.
[13] J. Redmon and A. Farhadi, "YOLO9000: Better, Faster, Stronger," In Proc of IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 6517-6525, 2017. doi: 10.1109/CVPR.2017.690.
[14] J. Redmon, A. Farhadi, “YOLOv3: An Incremental Improvement,” ArXiv abs/1804.02767, 2018.
[15] Z. Cao, G. Hidalgo, T. Simon, S. -E. Wei and Y. Sheikh, "OpenPose: Realtime Multi-Person 2D Pose Estimation Using Part Affinity Fields," In Proc of IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 43, no. 1, pp. 172-186, 2021. doi: 10.1109/TPAMI. 2019.2929257.
[16] S. Hershey, S. Chaudhuri, D. P. W. Ellis, J. F. Gemmeke, A. Jansen, C. Moore, M. Plakal, D. Platt, R. A. Saurous, B. Seybold, M. Slaney, R. Weiss, K. Wilson, "CNN architectures for large-scale audio classification," In Proc of 2017 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 2017, pp. 131-135, doi: 10.1109/ICASSP.2017.7952132.
[17] T. Ishibashi, Y. Nakao, and Y. Sugano, “Investigating audio data visualization for interactive sound recognition,” In Proc of the 25th International Conference on Intelligent User Interfaces (IUI), Association for Computing Machinery, New York, NY, USA, 67–77. doi:https://doi.org/10.1145/3377325.3377483
[18] Z. Wolkowicz, S. Brooks, V. Keselj, “Midivis: Visualizing Music Structure via Similarity Matrices,” In Proc of International computer music conference(ICMC), pp. 53-56, 2009.
[19] A.S. Cowen, X. Fang, D. Sauter, D. Keltner, “What music makes us feel: At least 13 dimensions organize subjective experiences associated with music across different cultures,” In Proc of the National Academy of Sciences, Vol. 117(4), pp. 1924-1934, 2020. doi: 10.1073/pnas. 1910704117.
[20] L. A. Gatys, A. S. Ecker, M. Bethge, “Image Style Transfer Using Convolutional Neural Networks,” In Proc of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 2414-2423, 2016.
[21] P. Isola, J. Zhu, T. Zhou and A. A. Efros, “Image-to-Image Translation with Conditional Adversarial Networks,” In Proc of IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 5967-5976, 2017. doi: 10.1109/CVPR.2017.632.
[22] Autodraw, https://www.autodraw.com/ (accessed June. 27, 2021)
[23] P.Isola, J. Zhu, T. Zhou, A. Efros, “Image-to-Image Translation with Conditional Adversarial Networks,” In Proc of IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 5967-5976, 2017. doi: 10.1109/CVPR.2017.632.
[24] A. Ramesh, M. Pavlov, G. Goh, S. Gray, C. Voss, A. Radford, M. Chen, I. Sutskever, “Zero-Shot Text-to-Image Generation,” ArXiv, abs/2102.12092. 2020.
[25] C. Weng, B. Curless, I. Kemelmacher, “Photo Wake-Up: 3D Character Animation From a Single Photo,” pp. 5901-5910, 2019. doi: 10. 1109/CVPR.2019.00606.
[26] J.P. Briot, G. Hadjeres, F. Pachet, “Deep Learning Techniques for Music Generation - A Survey,” Computational Synthesis and Creative Systems, Springer, 2017. [27] H. Dong, W. Hsiao, Li Yang, and Y. Yang, “MuseGAN: Multi-track Sequential Generative Adversarial Networks for Symbolic Music Generation and Accompaniment,” In Proc of the 32nd AAAI Conference on Artificial Intelligence (AAAI), 2018.
[28] L. Maaten and G.E. Hinton. Visualizing High-Dimensional Data Using t-SNE, Journal of Machine Learning Research, Vol 9, pp. 2579-2605, 2008.
[29] M. Wattenberg, F. Viegas, I. Johnson, “How to Use t-SNE Effective- ly,” Distill, 2016. http://doi.org/10.23915/distill.00002 (accessed June. 27, 2021)
[30] T-SNE visualization, https://lvdmaaten.github.io/tsne/ (accessed June. 27, 2021)
[31] Music visualization, https://www.ocf.berkeley.edu/~acowen/music. html# (accessed June. 27, 2021)

Comment


Editorial Office
1108, New building, 22, Teheran-ro 7-gil, Gangnam-gu, Seoul, Korea
Homepage: www.kibme.org TEL: +82-2-568-3556 FAX: +82-2-568-3557
Copyrightⓒ 2012 The Korean Institute of Broadcast and Media Engineers
All Rights Reserved