The YaCy Grid is the second-generation implementation of YaCy, a peer-to-peer search engine. A YaCy Grid installation consists of a set of micro-services which communicate with each other using a common infrastructure for data persistence. The task was to deploy the second-generation of YaCy Grid. To do so, we first had created a Dockerfile. This dockerfile should start the micro services such as rabbitmq, Apache ftp and elasticsearch in one docker instance along with MCP. The microservices perform following tasks:
- Apache ftp server for asset storage.
- RabbitMQ message queues for the message system.
- Elasticsearch for database operations.
To launch these microservices using Dockerfile, we referred to following documentations regarding running these services locally: https://github.com/yacy/yacy_grid_mcp/blob/master/README.md
For creating a Dockerfile we proceeded as follows:
We have created a start.sh file to start RabbitMQ and Apache FTP services. At the end, for compilation gradle run will be executed.
start.sh will first add username and then password. Then it will start RabbitMQ along with Apache FTP. For username and password, we have created a separate files to configure their properties during Docker run which can be found here:
- Configuration of FTP server: https://github.com/yacy/yacy_grid_mcp/blob/master/docker/config-ftp.properties
- Configuration of MCP service: https://github.com/yacy/yacy_grid_mcp/blob/master/docker/config-mcp.properties
The logic behind running all the microservices in one docker instance was: creating each container for microservice and then link those containers with the help of docker-compose.yml file.
The Dockerfile which we have created was corresponding to one image. Another image was elasticsearch which was linked to this Dockerfile. The latest version of elasticsearch image was already available on their site: https://www.elastic.co/guide/en/elasticsearch/reference/current/docker.html
We configured the docker-compose.yml file according to the reference link provided above. The docker-compose file can be found here: https://github.com/yacy/yacy_grid_mcp/blob/master/docker/docker-compose.yml
The source code for the implementation of whole structure can be found here: https://github.com/yacy/yacy_grid_mcp/tree/master/docker
- Dockerfile official documentation: https://docs.docker.com/engine/reference/builder/
- Docker Tutorial series by Romin Irani: https://rominirani.com/docker-tutorial-series-a7e6ff90a023