Code Collector

This README was written by ChatGPT, provided the collected code, generated by the Code Collector tool.

Table of Contents

• Roadmap
• Overview
• Features
• Installation
• Usage
• Contributing

Roadmap

• Coming up with a cool name
• Initial prototype
• Windows support
• Advanced format collecting (e.g., pdf, pptx, etc.)
  • PDF
• Additional output formats (e.g., JSON, XML)
• System-wide configuration
• Directory specific configuration
• Ignore file
  • Implement wild cards (e.g., *.pdf)
• Output size optimization (e.g., collapsing functions, code blocks, etc.)
• !!! Continuous code collection (watch mode) !!!
• Interactive CLI mode
• Packaging for Linux distributions

And more...

Overview

Code Collector is a Rust-based tool designed to collect and organize source code files from specified directories. It offers flexible configuration options to include or exclude specific paths or file formats.

The collected data is then written to a single .txt file, making it easy to supply to virtually any LLM or process in any other way.

Features

• Flexible Path Configuration: Specify multiple directories to scan for code files.
• Ignore File: Exclude files based on a specified ignore file.
• Format Filtering: Collect files based on specified formats/extensions.
• Ignore Paths: Exclude certain directories or files from the collection process.
• Output Options: Write the collected data to a specified output file.

Installation

Prerequisites

• Rust

Steps

1. Clone the Repository:

   git clone https://github.com/iwannabeacookie/CodeCollectoR

2. Navigate to the Project Directory:

   cd CodeCollectoR

3. Install the Project:

   cargo install --path .

4. Configure PATH (if needed):

   Ensure that the Cargo installation directory is in your system's PATH. The default installation directory is $HOME/.cargo/bin.

    echo $PATH | grep ".cargo/bin"

   If no output provided, proceed with the following steps.

   Edit your shell profile file (e.g., ~/.bashrc, ~/.zshrc, ~/.profile) and add the following line:

    export PATH="$HOME/.cargo/bin:$PATH"

   Changes will take effect after you restart your terminal or run source ~/.bashrc (or the corresponding file).

Usage

Code Collector can be configured via command-line arguments to specify paths, file formats, ignored directories, and the output file.

Command-Line Arguments

• -f, --formats <FORMATS>: Space-separated list of file formats/extensions to include (e.g., rs,py,js).
• -i, --ignore-paths <IGNORE_PATHS>: Space-separated list of paths to exclude from scanning.
• -o, --output <OUTPUT_FILE>: Path to the output file where collected data will be written.
• '-h, --help': Display help information.
• '-V, --version': Display version information.

.collectignore

The .collectignore file can be used to specify paths to ignore during the collection process. Each line in the file represents a path to be excluded. "#" are treated as comments and ignored. The filepaths in the .collectignore file are relative to the directory of the .collectignore and can stack up recursively.

# Ignore the tests directory
tests # or tests/

# Ignore the binary
target/out # Or place a .collectignore with "out" in the target directory

# Ignore single files
report/your_paper.pdf

Example

./code_collector \
  --paths ./src ./lib \
  --formats rs py \
  --ignore-paths ./src/tests ./lib/exclude \
  --output collected_code.txt

This command will:

• Scan the ./src and ./lib directories.
• Include only .rs and .py files.
• Ignore the ./src/tests and ./lib/exclude directories.
• Write the collected data to collected_code.txt.

Sample Output

Upon successful execution, the application will display a confirmation message:

Code collection completed. Output written to collected_code.txt

The collected_code.txt file will contain the structured data of the collected code files based on the specified configurations.

Project Structure:
CodeCollectoR/
├── src/
│   ├── collector/
│   │   ├── file_processor.rs
│   │   ├── directory_processor.rs
│   │   ├── mod.rs
│   │   └── path_handler.rs
│   ├── output/
│   │   ├── mod.rs
│   │   └── writer.rs
│   ├── formats/
│   │   ├── mod.rs
│   │   └── handler.rs
│   ├── lib.rs
│   ├── utils.rs
│   ├── cli.rs
│   ├── config.rs
│   └── main.rs
├── test_data/
│   ├── parco-d1-report.pdf
│   ├── src/
│   │   ├── a.rs
│   │   └── b.rs
│   └── othersrc/
│       └── a.rs
├── Cargo.lock
├── Cargo.toml
└── README.md

Code Files:

================
Filepath: /home/iwannabeacookie/Projects/CodeCollectoR/README.md
################
# Code Collector

*This README was written by ChatGPT, provided the collected code, generated by th
e Code Collector tool.*

## Table of Contents ...

Contributing

Contributions are welcome! If you have suggestions for improvements, bug fixes, or new features, please open an issue or submit a pull request.

Steps to Contribute

1. Fork the Repository

2. Create a New Branch

   git checkout -b feature/YourFeature

3. Commit Your Changes

   git commit -m "Add some feature"

4. Push to the Branch

   git push origin feature/YourFeature

5. Open a Pull Request

---

Thank you for using Code Collector! If you have any questions or need assistance, feel free to reach out.