Моделирование технологий обработки больших данных на локальной облачной платформе

Abstract

Предложена модель локальной облачной платформы c гибким перераспределением мощностей между рабочими нагрузками. Рассмотрена служба терминального доступа к рабочим столам и моделирование технологий обработки больших данных, развертывание архитектуры служб Apache Hadoop на виртуальном кластере, а также моделирование отдельных компонент поисковых систем.

Запропоновано модель локальної хмарної платформи з гнучким перерозподіленням потужностей між робочими навантаженнями. Розглянуто службу термінального доступу до робочих столів та моделювання технологій обробки великих даних, розгортання архітектури служб Apache Hadoop на віртуальному кластері, а також моделювання окремих компонент пошукових систем.

Introduction. The implementation of the operational local cloud platform model is considered which provide two services at the SaaS level. The first one relates to the issue of optimizing the organization of workplaces in the organization. The second one implements the model of the large data processing environment. The main issue is to solve the problem of combining heterogeneous tasks within a common environment and redistributing computing resources between them. Purpose. The purpose of this article is to build a model of a multi-purpose local cloud platform with a flexible redistribution of power between workloads and to consider the usages of two applications for the model: the service of terminal access to desktops and the Hadoop software platform for big data analysis. In addition, the modeling of the search engine element – the search robot – as the task of big data analysis. Methods. Methods of modeling and abstraction were used. Results. A functioning model of the local cloud platform, which provides a flexible mechanism for modeling and deploying platforms of a wide range of architectures, purpose and production, is proposed. The possibilities of using existing solutions of search robots are analyzed and our own experimental development is created. Both variants of search robots provide the necessary information in a suitable form for the work of consequential elements of search engines. Conclusion. The model is proposed as a general solution for the deployment of private local clouds in enterprises. One of the possible applications implemented based on a platform for big data analytics is the creation of a search engine.