You can visit the live project here.
Since the AI jobs processing the images to remove their background are long running tasks (5 to 10 seconds), we have to run them asynchronously. To achieve that I used a Redis instance that will be used as a queue to store the AI Jobs submitted by the Laravel Web App. The Web App will return return a 200 http code when a user upload an image and a Job will be queued. A mysql db is also used to track the jobs (failed jobs, job status).
Once the Job is sumbitted, a Worker service is listening to the queue and start processing it asynchronously. Once the AI job is done, the Worker service broadcasts an Event with the image url and the event is notified to the user thanks to the Websocket server.
I also wanted to build an app that can easily scale, especially an app that can scale horizontally. To achieve that I decoupled the storage layer from the Web app and from the Worker service. I used a separate Minio Cloud Storage service for that. Thanks to that the Worker Service and the Web app are stateless and can be replicated as needed and can the instances can live in separate servers.
In the production server there are 4 Workers services and the AI tasks are distributed equally in a round robbin manner among the different workers.
- Start the services used by the app: (mysql, redis, minio, reverb)
docker compose -f dev.docker-compose.yaml up -d - Run the migrations
php artisan migrateThis command will spin up a Websocket Reverb Server, a Redis for the queued Jobs a Minio server for storing the images and a Mysql server.
- Run the php server in a terminal
php artisan serve- In another terminal run the Vite server for hot reloading of Front assets (usefull for frontend dev)
npm run dev- Run a php queue worker to process Jobs
php artisan queue:work- Browse
https://localhost:8000
All the configuration variables used in the app are in the file .env at the root of the project.
sudo ./install_or_update.sh- Choose
localwhen environment is prompted. - Choose a
domain nameit will automatically add appropirate lines in/etc/hostsin order to route the requests toyour domain nametolocalhost. - Choose the
number of workersthat process the Background Removal tasks. - You can choose either or not to build the
workerimage. It does not depend ondomain nameand it takes a long time to build (it needs to download AI models) so you can skip the build if you did not change theworkercode after the lastworkerbuild. - Browse
http://yourdomain.com.
- Buy a VPS server with sufficiant RAM and CPU (at least 4GO RAM and 2 cores CPU)
- Buy a domain name
yourdomain - Manage DNS records of
yourdomainto point to your VPS ip address :yourdomain-> ip address- reverb.
yourdomain-> ip address - minio.
yourdomain-> ip address - adminio.
yourdomain-> ip address
sudo ./install_or_update.sh- Choose
productionwhen environment is prompted. - You can choose either or not to build the
workerimage. It does not depend ondomain nameand it takes a long time to build (it needs to download AI models) so you can skip the build if you did not change theworkercode after the lastworkerbuild. - Browse
https://yourdomain.com.
A CI/CD pipeline has been created thanks tot Github Actions. On new github release the 3 images: rembg, webserver and worker are build and pusehd to my Docker Hub registery. Then It connects to my VPS through ssh pull the new docker images and tells docker compose to recreate the containers with the new images. The CI/CD yaml file is located at .github/workflows/deploy.yaml.
gh release create <tag_name> --title "<release_title>" --notes "<release_notes>"Example:
gh release create v1.0.0 --title "First Release" --notes "a Handy app to automtically detect and remove background from images"- Use
actcommand. It will use the secret file:.secretsto fill the secrets in thedeploy.yamlmanifest.
act- Test a specific Job
act -j build- SEO optimization with some Laravel Package maybe ?
- Disable Session cookie for unauthenticated users (the app does not track user) so they are useless. Then it will allow to cache the home page at nginx level or Cache it using a CDN. It will improve home page response time which is good for SEO.
- Video support
- Create Kubernetes manifests and a Helm package to allow autoscaling horizontally in a cluster of nodes or maybe use Swarm ?