pip:
- Clone the repository from your version-control system.
- Create a virtual environment with your tool/distro of choice.
- Run the following:
Components
1. DLConfig
DLConfig reads configurations (e.g., API keys) via environment variables or .env files. This enables dynamic adaptation across deployment environments.
2. DataLinksAPI
DataLinksAPI handles interactions with the API. Using it, you can:
- Ingest data
- Query or retrieve data with complex parameters
- Manage namespaces
3. Inference Workflow
Use a chain of inference and validation steps defined through classes likeProcessUnstructured, Normalize, and Validate to automate data preparation workflows.
4. Entity Resolution
Supports multiple resolution strategies, configurable viaMatchTypeConfig:
5. Loaders
Abstract base loaders (e.g.,JSONLoader) allow seamless data ingestion from custom file formats like .json.
6. Parametrize LLms
You can choose the model and provider to be used in inference steps (eg.:ProcessUnstructured, Normalize, Validate).