Setup & Installation
What This Skill Does
Python SDK for Azure Data Lake Storage Gen2, which provides a hierarchical file system built on Azure Blob Storage. Supports file and directory operations, access control lists, and large file uploads via chunked append and flush patterns.
The hierarchical namespace and ACL support mean you can model real directory structures and fine-grained permissions directly in storage, which flat object stores like S3 or plain Blob Storage require workarounds to approximate.
When to use it
- Uploading Spark or Databricks output files into organized directory hierarchies
- Setting user-level ACL permissions on a shared analytics folder
- Listing all files recursively under a specific partition path
- Appending log chunks to a file without overwriting existing data
- Moving processed files to an archive directory after pipeline completion