Thanks to visit codestin.com
Credit goes to github.com

Skip to content

pratikbin/go-s3-bulk-del

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

3 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

sh-del-all-jpeg

A high-performance Go utility to bulk delete all .jpg files from an S3-compatible object storage bucket.

Features

  • Concurrent Deletion: Uses a worker pool (default: 20 workers) to delete objects in parallel for maximum throughput.
  • Batch Processing: Deletes objects in batches of up to 1000 keys to minimize API requests and improve speed.
  • S3 Compatible: Works with AWS S3 and other S3-compatible providers (e.g., MinIO, DigitalOcean Spaces, Hetzner Object Storage).
  • Filtered Deletion: Specifically targets and deletes files ending in .jpg (case-insensitive), leaving other files untouched.

Prerequisites

  • Go 1.25 or higher
  • Access to an S3-compatible object storage bucket

Configuration

The application is configured entirely via environment variables:

Variable Description Default Required
S3_ACCESS_KEY Your S3 Access Key ID - Yes
S3_SECRET_KEY Your S3 Secret Access Key - Yes
S3_BUCKET Target bucket name argus No
S3_REGION Bucket region fsn1 No
S3_ENDPOINT S3 Endpoint URL https://fsn1.your-objectstorage.com No

Usage

  1. Clone the repository (if applicable) or navigate to the project directory.

  2. Install dependencies:

    go mod tidy
  3. Run the tool:

    You can run it directly with go run:

    export S3_ACCESS_KEY="your-access-key"
    export S3_SECRET_KEY="your-secret-key"
    export S3_BUCKET="my-target-bucket"
    # Optional overrides:
    # export S3_ENDPOINT="https://s3.amazonaws.com"
    # export S3_REGION="us-east-1"
    
    go run main.go

    Alternatively, build the binary:

    go build -o go-s3-bulk-del main.go
    
    # Run with env vars
    S3_ACCESS_KEY="key" S3_SECRET_KEY="secret" ./go-s3-bulk-del

⚠️ Warning

This tool performs destructive actions. It will permanently delete ALL files ending in .jpg in the specified bucket without confirmation prompts. Ensure you have the correct bucket selected and have backups if necessary before running.

About

No description, website, or topics provided.

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

 
 
 

Contributors

Languages