General-purpose command-line utilities for data processing, file management, and system administration.
A collection of practical command-line utilities for common tasks in data processing, file operations, and system management. These tools are designed to be simple, efficient, and composable with standard Unix workflows.
# Clone repository
git clone https://github.com/olympus-terminal/tools.git
cd tools
# Make scripts executable
chmod +x *.sh *.py *.pl
# Add to PATH (optional)
export PATH="$PATH:$(pwd)"
break-to-100s.py - Split data into chunks of 100 entries
python break-to-100s.py large_dataset.txt
delim_to_tab_UTF8 - Convert any delimiter to tab-separated values
./delim_to_tab_UTF8 < data.csv > data.tsv
only_alpha - Extract only alphabetic characters from input
./only_alpha < mixed_text.txt > letters_only.txt
get_IDs - Extract IDs from formatted text
./get_IDs input_file.txt > ids.txt
move_large_files.sh - Efficiently move large files with progress
./move_large_files.sh /source/dir /dest/dir "*.bam"
osxrename.pl - Perl-based batch file renaming (macOS compatible)
./osxrename.pl 's/old_pattern/new_pattern/g' *.txt
filter_ckpts.sh - Filter and manage checkpoint files
./filter_ckpts.sh checkpoints/ --keep-every 10
opposite_day.sh - Reverse file selection logic
./opposite_day.sh pattern files/
pdf_to_txt_argv.py - Convert PDF files to plain text
python pdf_to_txt_argv.py document.pdf output.txt
mp3xtractr.py - Extract metadata from MP3 files
python mp3xtractr.py music_library/ > metadata.csv
neural_tts.py - Text-to-speech synthesis utility
python neural_tts.py "Text to convert to speech" output.wav
CatchAndSlot.sh - Execute commands with proper error handling
./CatchAndSlot.sh long_running_command
distribute_ex.sh - Distribute files across multiple systems
./distribute_ex.sh file_list.txt server1 server2 server3
clone_imprint_distribute - Clone and distribute configurations
./clone_imprint_distribute template/ target_systems.txt
download-update.sh - Automated download and update script
./download-update.sh package_name
extract_zip_having_dups.sh - Extract ZIP files with duplicate handling
./extract_zip_having_dups.sh archive.zip output_dir/
force_unzip.sh - Force extraction of damaged archives
./force_unzip.sh corrupted.zip output_dir/
optimized_rsync.sh - Rsync wrapper with optimization
./optimized_rsync.sh /source/dir/ user@host:/dest/dir/
getFromNCBI-gene.py - Fetch gene data from NCBI
python getFromNCBI-gene.py gene_list.txt > gene_data.txt
cid_make_unitigs - Process unitigs from assembly
./cid_make_unitigs assembly.fa > unitigs.fa
interleave.sh - Interleave paired-end sequencing reads
./interleave.sh reads_R1.fq reads_R2.fq > interleaved.fq
combine-ig-headers.sh - Combine immunoglobulin sequence headers
./combine-ig-headers.sh sequences.fa > combined.fa
awktag.sh - AWK-based string tagging
./awktag.sh pattern input.txt
betterthanbasename - Enhanced basename functionality
./betterthanbasename /path/to/file.tar.gz .tar.gz
# Convert CSV to TSV, extract IDs, process in chunks
cat data.csv | \
./delim_to_tab_UTF8 | \
./get_IDs | \
python break-to-100s.py
# Rename all files matching pattern
./osxrename.pl 's/IMG_/photo_/g' *.jpg
# Move large files with progress
./move_large_files.sh raw_data/ processed_data/ "*.fastq.gz"
# Distribute configuration to multiple servers
./distribute_ex.sh config_files.txt web1 web2 web3
# Execute with error handling
./CatchAndSlot.sh ./critical_backup.sh
Most tools require only standard Unix utilities. Specific requirements:
- Python scripts: Python 3.6+
- Perl scripts: Perl 5.10+
- C programs: Compile with
gcc only_alpha.c -o only_alpha
Python dependencies (if needed):
pip install beautifulsoup4 requests pdftotext
# Clone repository
git clone https://github.com/olympus-terminal/tools.git
cd tools
# Make all scripts executable
find . -name "*.sh" -o -name "*.py" -o -name "*.pl" | xargs chmod +x
# Compile C programs
gcc only_alpha.c -o only_alpha
# Add to PATH
echo 'export PATH="$PATH:'$(pwd)'"' >> ~/.bashrc
source ~/.bashrc
Contributions welcome. Please:
- Fork the repository
- Create a feature branch
- Add your tool with usage documentation
- Submit a pull request
MIT License - see LICENSE file for details.
- Issues: GitHub Issues
- Author: @olympus-terminal