Year: 2020
Type: conference-paper
Abstract: Machine learning technology is indispensable for big data. In machine learning, data on a large scale can improve the accuracy of the model. However, complex machine learning algorithms require the key technology of distributed memory computing in time and performance. Big data memory computing can implement the parallel operation of the algorithm, which is conducive to the processing of big data sets by the machine learning algorithm. Hence, a nonlinear machine learning algorithm implemented in the big data memory environment is proposed in this paper, where data compression, biased sampling or loading based on the implementation is optimized. To fully configure resources for the script running by batch, we also implemented a machine learning framework to schedule the optimized algorithm mentioned above. The experimental results showed that the mean error of the three algorithms after optimization was reduced by 40%, and the mean time was reduced by 90%.
Author: HaiYan Hao
Institution: Hainan College of Software Technology
Language: English
Subfield: Information Systems
Field: Computer Science
Domain: Physical Sciences
Open Access status: diamond
OpenAlex ID: W3035922882
