Thanks to visit codestin.com
Credit goes to github.com

Skip to content

Commit aad11c2

Browse files
napetrovcursoragentVika-F
authored
Introducing AGENTS.md and Copilot instructions (#3335)
* initial version * initial version * Enhance copilot onboarding with PR review focus and build system priorities - Strengthen PR review as PRIMARY GOAL across all instruction files - Add visual indicators (🔴🟡🟢) for build system priorities - Emphasize Make as CRITICAL for production builds - Create comprehensive PR review templates and checklists - Streamline examples for quick review scenarios - Add cross-reference navigation improvements - Include quick decision guides for common scenarios - Enhance CI workflow guidance with PR review focus Critical improvements: - Make compatibility is CRITICAL (verify FIRST) - CMake integration is IMPORTANT (verify SECOND) - Bazel testing is DEVELOPMENT (verify THIRD) - PR Review assistance is the PRIMARY goal * Add comprehensive improvement summary document - Document all critical issues resolved - Detail specific improvements implemented - Include quality metrics improvements - Provide expected impact analysis - Outline next steps for validation - Create reference for future improvements * Remove PR_REVIEW_TEMPLATES.md and add comprehensive coding guidelines - Remove PR_REVIEW_TEMPLATES.md file as requested - Create comprehensive coding-guidelines.md based on dev guide analysis - Update README.md to include new coding guidelines file - Enhance general.md with coding standards references - Integrate coding standards into PR review checklists - Ensure consistency across all instruction files New coding guidelines include: - Coding style standards and formatting - Naming conventions for all code elements - Declaration order and class organization - Programming guidelines and best practices - SDL requirements for security and performance - Comments and documentation standards - Comprehensive PR review coding checklist * Remove IMPROVEMENT_SUMMARY.md file as requested * cleaned version * fixing copyrights * fixing copyrights * fixing copyrights * Restructure build-system.md and cleaning license headers * Cleaning headers from AGENTS.md * Adressing comments * Path 2 for context * Cleanup wording * improving docs * Updating local knowledge * fixing links * reducing instructions size * removed extra files * adressing feedback * Adressing comments * adressing code review comments * adressing code review comments * Additions and fixes in daal/AGENTS.md * Additions to cpp/AGENTS.md * Additions to oneapi/AGENTS.md --------- Co-authored-by: Cursor Agent <[email protected]> Co-authored-by: Victoriya Fedotova <[email protected]>
1 parent 773af96 commit aad11c2

16 files changed

+2221
-0
lines changed

‎.ci/AGENTS.md‎

Lines changed: 112 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,112 @@
1+
# oneDAL CI Infrastructure Documentation
2+
3+
## Overview
4+
5+
This document describes the CI infrastructure for oneDAL (Intel Data Analytics Library)
6+
## Directory Structure
7+
8+
### `.ci` Folder
9+
10+
The `.ci` folder contains the core CI infrastructure scripts and configurations organized into three main subdirectories:
11+
12+
#### `.ci/env/` - Environment Setup Scripts
13+
- **`apt.sh`** - Package installation script for Ubuntu/Debian systems with functions for:
14+
- Intel OneAPI toolkit components (DPC++, TBB, DPL, MKL)
15+
- Development tools (clang-format, editorconfig-checker)
16+
- Base development packages and dependencies
17+
- **`bazelisk.sh`** - Bazel build system setup and installation
18+
- **`editorconfig-checker.sh`** - EditorConfig compliance checker installation
19+
- **`environment.yml`** - Conda environment specification
20+
- **`openblas.sh`** - OpenBLAS library installation and configuration
21+
- **`openrng.sh`** - OpenRNG backend setup for random number generation
22+
- **`tbb.sh`** / **`tbb.bat`** - Intel TBB (Threading Building Blocks) setup for Linux/Windows
23+
- **`riscv64-clang-crosscompile-toolchain.cmake`** - RISC-V cross-compilation toolchain configuration
24+
25+
#### `.ci/pipeline/` - CI Pipeline Definitions
26+
- **`ci.yml`** - Main Azure DevOps pipeline configuration with:
27+
- Multi-platform build matrices (Linux, Windows)
28+
- Compiler configurations (GNU, Clang, Intel)
29+
- Build targets (daal, onedal_c)
30+
- Testing and validation jobs
31+
- Artifact publishing
32+
- **`docs.yml`** - Documentation build and deployment pipeline
33+
34+
#### `.ci/scripts/` - Build and Test Scripts
35+
- **`build.sh`** / **`build.bat`** - Cross-platform build orchestration with support for:
36+
- Multiple compilers (gnu, clang, icx)
37+
- Architecture optimizations (AVX2, etc.)
38+
- Backend configurations (MKL, reference implementations)
39+
- Cross-compilation capabilities
40+
- **`test.sh`** / **`test.bat`** - Comprehensive testing framework execution
41+
- **`clang-format.sh`** - Code formatting verification
42+
- **`describe_system.sh`** - System information collection for debugging
43+
- **`abi_check.sh`** - ABI compatibility verification
44+
- **`install_basekit.bat`** - Intel OneAPI Base Toolkit installation for Windows
45+
- **`collect_opencl_rt.ps1`** - OpenCL runtime collection script
46+
47+
### `.github/workflows/` - GitHub Actions Workflows
48+
49+
#### Core CI Workflows
50+
- **`ci.yml`** - Main CI pipeline for x86 platforms with DPC++ builds
51+
- **`ci-aarch64.yml`** - AArch64 (ARM64) specific CI pipeline
52+
- **`nightly-build.yml`** / **`nightly-test.yml`** - Automated nightly builds and testing
53+
54+
#### Specialized Workflows
55+
- **`docker-validation-ci.yml`** / **`docker-validation-nightly.yml`** - Container-based validation
56+
- **`docs-release.yml`** - Documentation deployment and release management
57+
- **`label-enforcement.yml`** - PR labeling automation
58+
- **`pr-checklist.yml`** - Pull request compliance verification
59+
- **`renovate-validation.yml`** - Dependency update validation
60+
- **`skywalking-eyes.yml`** - License header compliance checking
61+
- **`slack-pr-notification.yml`** - Team notification system
62+
- **`openssf-scorecard.yml`** - Security scorecard assessment
63+
64+
## CI/CD Architecture
65+
66+
### Multi-Platform Strategy
67+
The oneDAL CI infrastructure supports:
68+
- **Architectures**: x86-64, AArch64, RISC-V
69+
- **Operating Systems**: Ubuntu (multiple versions), Windows Server 2022
70+
- **Compilers**: GNU GCC, Clang, Intel DPC++/ICX
71+
- **Build Systems**: Make, Bazel
72+
- **Hardware**: CPU and GPU (Intel) testing
73+
74+
### CI Platform Integration
75+
- **GitHub Actions**: Primary CI/CD platform for public workflows
76+
- **Azure DevOps**: Extended validation and internal testing
77+
- **Mergify**: Automated merge management
78+
- **Renovate**: Dependency update automation
79+
- **Codefactor**: Code quality analysis
80+
81+
### Build Matrix Configuration
82+
The CI system employs comprehensive build matrices covering:
83+
- **Instruction Sets**: AVX2, AVX-512, and architecture-specific optimizations
84+
- **Backends**: Intel MKL, OpenBLAS reference implementations
85+
- **Threading**: Intel TBB, OpenMP
86+
- **Random Number Generation**: Intel MKL RNG, OpenRNG
87+
88+
### Quality Assurance
89+
- **Code Formatting**: clang-format enforcement
90+
- **License Compliance**: Automated header checking
91+
- **ABI Compatibility**: Binary interface stability validation
92+
- **Security**: OpenSSF Scorecard integration
93+
- **Documentation**: Automated doc generation and validation
94+
95+
## Usage Guidelines
96+
97+
### Local Development
98+
Developers can leverage the CI scripts locally:
99+
```bash
100+
# Set up development environment
101+
.ci/env/apt.sh dev-base
102+
.ci/env/apt.sh mkl
103+
104+
# Build the library
105+
.ci/scripts/build.sh --compiler gnu --optimizations avx2 --target daal
106+
107+
# Run tests
108+
.ci/scripts/test.sh
109+
```
110+
111+
### Internal CI
112+
Internal CI integration done in separate repository, though checks are enforced in PRs.

‎.github/.licenserc.yaml‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -70,12 +70,14 @@ header:
7070
- '.ci/pipeline/**/*.yml'
7171
- '.ci/env/environment.yml'
7272
- '.github/workflows/*.yml'
73+
- '.github/instructions/*.md'
7374
- '.github/.mergify.yml'
7475
- '.github/CODEOWNERS'
7576
- '.github/pull_request_template.md'
7677
- '.github/renovate.json'
7778
# Specific files
7879
- 'LICENSE'
80+
- 'AGENTS.md'
7981
- 'third-party-programs-mkl.txt'
8082
- 'third-party-programs.txt'
8183
- 'deploy/local/config.txt'

‎.github/instructions/build-systems.instructions.md‎

Lines changed: 149 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,149 @@
1+
---
2+
applyTo: ["**/makefile", "**/Makefile", "**/BUILD", "**/BUILD.bazel", "**/*.bazel", "**/*.mk", "**/CMakeLists.txt"]
3+
---
4+
5+
# Build Systems Instructions for GitHub Copilot
6+
7+
## Build System Priority
8+
9+
1. **Make** - Primary production builds, core library
10+
2. **CMake** - End-user integration, examples
11+
3. **Bazel** - Development, testing, new features
12+
13+
## 🚀 Essential Commands
14+
15+
### Make (Production)
16+
```bash
17+
# Build everything
18+
`make`
19+
20+
# Platform-specific builds
21+
`make PLAT=lnx32e COMPILER=icx`
22+
`make PLAT=win32e COMPILER=vc`
23+
24+
# CPU targets
25+
`make REQCPU="sse42 avx2 avx512"`
26+
27+
# Backend selection
28+
`make BACKEND_CONFIG=mkl` # Intel MKL (default)
29+
`make BACKEND_CONFIG=ref` # Reference/OpenBLAS
30+
```
31+
32+
### Bazel (Development)
33+
```bash
34+
# Build targets
35+
`bazel build //cpp/oneapi/dal:core`
36+
`bazel build //examples/daal/cpp:association_rules`
37+
38+
# Test targets
39+
`bazel test //cpp/oneapi/dal:tests`
40+
`bazel test --config=dpc //cpp/oneapi/dal:tests` # GPU tests
41+
```
42+
43+
### CMake (Integration)
44+
```bash
45+
# Configure and build
46+
`cmake -B build -S . -DCMAKE_BUILD_TYPE=Release`
47+
`cmake --build build --parallel`
48+
```
49+
50+
## 🏗️ CPU Architecture Support
51+
52+
- **x86-64**: sse2, sse42, avx2, avx512 (Intel/AMD)
53+
- **ARM64**: sve (ARM Scalable Vector Extension)
54+
- **RISC-V**: rv64 (RISC-V 64-bit)
55+
56+
### CPU Dispatch Pattern
57+
```cpp
58+
// Runtime CPU detection and optimal code selection
59+
template <typename algorithmFPType, CpuType cpu>
60+
services::Status compute(/* parameters */) {
61+
// CPU-specific optimized implementation
62+
}
63+
```
64+
65+
## 🔧 Platform Configuration
66+
67+
### Linux (lnx32e)
68+
```makefile
69+
PLAT := lnx32e
70+
COMPILER := icx # Intel oneAPI C++/DPC++
71+
COMPILER := gnu # GCC
72+
COMPILER := clang # Clang
73+
```
74+
75+
### Windows (win32e)
76+
```makefile
77+
PLAT := win32e
78+
COMPILER := vc # Microsoft Visual C++
79+
COMPILER := icx # Intel oneAPI
80+
```
81+
82+
### Required Environment
83+
```bash
84+
# Intel oneAPI (for DPC++)
85+
export ONEAPI_ROOT=/path/to/oneapi
86+
export PATH=$ONEAPI_ROOT/compiler/latest/linux/bin:$PATH
87+
88+
# TBB (required)
89+
export TBBROOT=/path/to/tbb
90+
91+
# MKL (if using MKL backend)
92+
export MKLROOT=/path/to/mkl
93+
```
94+
95+
## 🎯 Bazel Build Rules
96+
97+
### Algorithm Module Pattern
98+
```python
99+
# DAAL module
100+
daal_module(
101+
name = "kmeans",
102+
features = [ "c++17" ],
103+
cpu_defines = {
104+
"sse2": [ "DAAL_CPU=sse2" ],
105+
"avx2": [ "DAAL_CPU=avx2" ],
106+
"avx512": [ "DAAL_CPU=avx512" ],
107+
},
108+
)
109+
110+
# oneAPI module
111+
dal_module(
112+
name = "kmeans",
113+
compile_as = ["c++", "dpc++"], # CPU and GPU
114+
)
115+
```
116+
117+
### Test Configuration
118+
```python
119+
dal_test_module(
120+
name = "core_test",
121+
compile_as = ["c++", "dpc++"],
122+
dal_deps = [":core"],
123+
)
124+
```
125+
126+
## 🔍 Common Issues
127+
128+
### Build Failures
129+
- **C++17 compliance**: Ensure GCC 7+, Clang 6+, MSVC 2017+
130+
- **Missing dependencies**: Check TBB, MKL installation
131+
- **CPU targets**: Verify CPU flags match target architecture
132+
133+
### Platform Differences
134+
- **Windows**: Use `vc` compiler for native Windows builds
135+
- **Linux**: Prefer `icx` for Intel optimizations, `gnu` for compatibility
136+
- **ARM**: Use `clang` compiler, set `PLAT=lnxarm`
137+
138+
## 🎯 Critical Rules
139+
140+
- **Make**: Primary for production builds and CI/CD
141+
- **Bazel**: Development workflow and new feature development
142+
- **CMake**: User integration, not primary build system
143+
- **CPU Dispatch**: Always implement multi-architecture support
144+
- **Dependencies**: TBB required, MKL preferred for performance
145+
146+
## 🔗 References
147+
148+
- **[general.instructions.md](general.instructions.md)** - Repository overview
149+
- **[cpp-coding-guidelines.instructions.md](cpp-coding-guidelines.instructions.md)** - C++ standards

0 commit comments

Comments
 (0)