The IoT Based Big Data Platform for Intelligent Manufacturing constitutes an important part in the infrastructure construction project of our Advanced Institute of Engineering Science for Intelligent Manufacturing. Powered by the high-performance hardware and the powerful Spark framework, our platform will be able to support the whole workflow of data processing from sampling to application. The platform is highly general and could serve customs from different industries with merely some customized setting in hardware and software. The figure below shows the basic architecture of the platform using the example of the Materials Genome Project. The frames in red indicate the project based customized setting (basic resources, Big Data application…), the rest forms the general architecture of the platform.
The platform integrates the basic components for Big Data processing. On the data sampling layer, Sqoop functions as the data switching system and processes different kinds of structured and non-structured data. On the other side, HDFS, YARN, Ambrai, Oozie, ZooKepper and NoSQL serve on the data storage and management layer to store, manage et schedule data. On the application support layer and application layer, the platform provides the following components: the BI tools for data mining, statement service etc., the distributed memory computing framework Spark, the machine learning library Mlib, the deep learning framework Tensorflow, the Stream Processing tool Spark Streaming, the Graph Calculation tool GraphX, the notebook tool Spark Notebook and the Big Data query system sparkSQL. All these are pre-installed to largely simplify the complicated operations such as Cluster Configuration/Installation and maintenance. Users could easily realize the resource service management, the billing management, the maintenance management, the operation monitoring and controlling etc. on the platform. Besides, with the integrated and customized applications, users would be able to achieve the demanded Big Data applications, for the example of the Materials Genome Project, one can realize the tasks like material performance analysis, new material research and development, industry analysis and workflow generation etc.
The Advanced Institute of Engineering Science for Intelligent Manufacturing of Guangzhou University possesses the hardwares of high performance. The IoT Based Big Data Platform for Intelligent Manufacturing that it makes with the state-of-the art technique of Big Data has the strengths of high-performance, lightweight and extensibility. The platform simplifies largely the configuration for the users and allows the enterprises to extend the basic architecture according to their real needs and create a more powerful customized platform.
