> ## Documentation Index
> Fetch the complete documentation index at: https://docs.datalinks.com/llms.txt
> Use this file to discover all available pages before exploring further.

# Ingest data

> Load data into the specified dataset.



## OpenAPI

````yaml /api-reference/openapi.yml post /ingest/{namespace}/{datasetName}
openapi: 3.0.3
info:
  title: DataLinks
  version: 2.2.3
servers:
  - url: https://api.datalinks.com/api/v1
    description: production server
security:
  - bearerAuth: []
  - openId:
      - implicit
paths:
  /ingest/{namespace}/{datasetName}:
    post:
      tags:
        - ingest
      summary: Ingest data
      description: Load data into the specified dataset.
      operationId: ingestData
      parameters:
        - name: namespace
          in: path
          description: Namespace for the dataset.
          required: true
          schema:
            type: string
          example: cinematography
        - name: datasetName
          in: path
          description: Name of the dataset.
          required: true
          schema:
            type: string
          examples:
            movies:
              summary: Movies
              value: movies
            actors:
              summary: Actors
              value: actors
      requestBody:
        content:
          application/json:
            schema:
              $ref: '#/components/schemas/IngestRequest'
            examples:
              movies:
                $ref: '#/components/examples/moviesExample'
              actors:
                $ref: '#/components/examples/actorsExample'
              unstructured:
                $ref: '#/components/examples/unstructuredExample'
        required: true
      responses:
        '200':
          description: Returns the ingestion result.
        '401':
          $ref: '#/components/responses/Unauthorized'
components:
  schemas:
    IngestRequest:
      required:
        - data
      type: object
      properties:
        data:
          type: array
          items:
            type: object
            additionalProperties:
              type: string
        infer:
          $ref: '#/components/schemas/IngestRequestInfer'
        link:
          $ref: '#/components/schemas/IngestLink'
        curate:
          type: boolean
          nullable: true
        dataDescription:
          $ref: '#/components/schemas/DataDescription'
        schemaDefinition:
          $ref: '#/components/schemas/SchemaDefinition'
        additionalInstructions:
          type: string
          description: Additional instructions to guide the AI system during ingestion.
    IngestRequestInfer:
      type: object
      properties:
        steps:
          type: array
          items:
            $ref: '#/components/schemas/IngestRequestInferStep'
    IngestLink:
      type: object
      properties:
        ExactMatch:
          nullable: true
          anyOf:
            - $ref: '#/components/schemas/ExactMatchConfig'
            - $ref: '#/components/schemas/ResolutionParameters'
        GeoMatch:
          nullable: true
          anyOf:
            - $ref: '#/components/schemas/GeoMatchConfig'
            - $ref: '#/components/schemas/ResolutionParameters'
    DataDescription:
      maxLength: 10000
      type: string
      description: >-
        A description of the dataset to help the AI system understand the
        context.
      example: Employee dataset containing demographics and compensation.
    SchemaDefinition:
      type: object
      additionalProperties:
        type: string
      description: >-
        A schema definition mapping field names to descriptions, to guide the AI
        system in structuring the extracted data.
      example:
        id: unique employee identifier
        name: full employee name
        age: employee age in years
        department: work department
        salary: annual salary in USD
    ErrorResponse:
      required:
        - error_code
        - message
      type: object
      properties:
        error_code:
          type: string
          example: UNAUTHORIZED
        error_message:
          type: string
          example: ''
        message:
          type: string
          example: Missing authentication token
        sub_errors:
          type: array
          items:
            required:
              - code
              - message
            type: object
            properties:
              code:
                type: string
              message:
                type: string
    IngestRequestInferStep:
      type: object
      discriminator:
        propertyName: type
        mapping:
          table:
            $ref: '#/components/schemas/TableStep'
          rows:
            $ref: '#/components/schemas/RowsStep'
          normalise:
            $ref: '#/components/schemas/NormaliseStep'
          validate:
            $ref: '#/components/schemas/ValidateStep'
          reverseGeo:
            $ref: '#/components/schemas/ReverseGeoStep'
          namedEntities:
            $ref: '#/components/schemas/NamedEntitiesStep'
      anyOf:
        - $ref: '#/components/schemas/TableStep'
        - $ref: '#/components/schemas/RowsStep'
        - $ref: '#/components/schemas/NormaliseStep'
        - $ref: '#/components/schemas/ValidateStep'
        - $ref: '#/components/schemas/ReverseGeoStep'
        - $ref: '#/components/schemas/NamedEntitiesStep'
    ExactMatchConfig:
      type: object
      properties:
        minDistinct:
          type: number
        minVariation:
          type: number
      nullable: true
    ResolutionParameters:
      type: object
      properties:
        bidirectional:
          type: boolean
        targetColumns:
          type: array
          items:
            type: string
        ignorePrivateColumns:
          type: boolean
    GeoMatchConfig:
      type: object
      properties:
        distance:
          type: number
        distanceUnit:
          type: string
      nullable: true
    TableStep:
      required:
        - deriveFrom
        - type
      type: object
      properties:
        type:
          type: string
          enum:
            - table
        deriveFrom:
          type: string
          description: Name of the source column to extract from.
        additionalInstructions:
          type: string
          description: >-
            Additional instructions to guide the AI system for higher accuracy
            during extraction.
          example: >-
            This text contains a table comma separated, but instead of line
            breaks we are using a ';'.
        model:
          type: string
          description: The LLM model to use for this step.
        provider:
          type: string
          description: The LLM provider to use for this step.
        replaceOriginalColumn:
          type: boolean
          description: >-
            When true, the source column specified in deriveFrom is removed from
            the output after extraction.
          default: false
    RowsStep:
      required:
        - deriveFrom
        - type
      type: object
      properties:
        type:
          type: string
          enum:
            - rows
        deriveFrom:
          type: string
          description: Name of the column that contains the JSON content.
    NormaliseStep:
      required:
        - mode
        - type
      type: object
      properties:
        type:
          type: string
          enum:
            - normalise
            - normalize
        mode:
          type: string
          description: Normalisation mode.
          enum:
            - all-in-one
            - field-by-field
            - embeddings
        targetCols:
          type: object
          additionalProperties:
            type: string
          description: Target columns with their descriptions.
        dataDescription:
          $ref: '#/components/schemas/DataDescription'
        model:
          type: string
          description: The LLM model to use for this step.
        provider:
          type: string
          description: The LLM provider to use for this step.
    ValidateStep:
      required:
        - columns
        - mode
        - type
      type: object
      properties:
        type:
          type: string
          enum:
            - validate
        mode:
          type: string
          description: Validation mode.
          enum:
            - regex
            - rows
            - fields
        columns:
          type: array
          description: Columns to validate.
          items:
            type: string
        model:
          type: string
          description: The LLM model to use for this step.
        provider:
          type: string
          description: The LLM provider to use for this step.
    ReverseGeoStep:
      required:
        - type
      type: object
      properties:
        type:
          type: string
          enum:
            - reverseGeo
        deriveFrom:
          type: string
          description: Name of the location column.
    NamedEntitiesStep:
      required:
        - type
      type: object
      properties:
        type:
          type: string
          enum:
            - namedEntities
  examples:
    moviesExample:
      summary: Movies
      value:
        data:
          - name: Braveheart
            year: 1995
            budget: 72000000
            director: Mel Gibson
          - name: Argo
            year: 2012
            budget: 44500000
            director: Ben Affleck
          - name: A Quiet Place
            year: 2018
            budget: 17000000
            director: John Krasinski
          - name: Dances with Wolves
            year: 1990
            budget: 22000000
            director: Kevin Costner
          - name: Unforgiven
            year: 1992
            budget: 14000000
            director: Clint Eastwood
          - name: The Postman
            year: 1997
            budget: 80000000
            director: Kevin Costner
        link:
          ExactMatch: null
    actorsExample:
      summary: Actors
      value:
        data:
          - name: Mel Gibson
            age: 68
            nationality: American
          - name: Sophie Marceau
            age: 57
            nationality: French
          - name: Ben Affleck
            age: 52
            nationality: American
          - name: Bryan Cranston
            age: 68
            nationality: American
          - name: John Krasinski
            age: 45
            nationality: American
          - name: Emily Blunt
            age: 41
            nationality: British
          - name: Kevin Costner
            age: 69
            nationality: American
          - name: Mary McDonnell
            age: 72
            nationality: American
          - name: Clint Eastwood
            age: 94
            nationality: American
          - name: Morgan Freeman
            age: 86
            nationality: American
          - name: Kevin Costner
            age: 69
            nationality: American
          - name: Will Patton
            age: 70
            nationality: American
        link:
          ExactMatch: null
    unstructuredExample:
      summary: Example of extracting rows from unstructured data
      value:
        data:
          - text: >-
              Meryl Streep, known for her incredible acting, has starred in
              unforgettable films like The Devil Wears Prada and Sophie's
              Choice. She's won three Oscars over her illustrious career. Anne
              Hathaway shared the screen with her in The Devil Wears Prada and
              later shone in Les Misérables, earning her own Oscar.


              Leonardo DiCaprio, a household name, is celebrated for roles in
              Titanic, Inception, and The Revenant. His long-awaited Oscar win
              came for The Revenant. His Titanic co-star Kate Winslet, who also
              has an Oscar to her name, delivered powerful performances in The
              Reader and of course, Titanic.


              Tom Hanks, the quintessential leading man, gave us unforgettable
              performances in Forrest Gump, Cast Away, and Saving Private Ryan.
              He's a two-time Oscar winner. Robin Wright, who starred alongside
              him in Forrest Gump, is also well-remembered for her role in The
              Princess Bride.


              Cate Blanchett, a two-time Oscar winner, mesmerized audiences in
              films like Blue Jasmine, Carol, and Elizabeth. Rooney Mara, her
              co-star in Carol, also made a mark with her performance in The
              Girl with the Dragon Tattoo.


              Denzel Washington, one of the most respected actors in Hollywood,
              has won two Oscars for his work in Training Day and Glory. He also
              delivered a heartfelt performance in Fences, where he starred
              alongside Viola Davis. Davis herself is a powerhouse of talent,
              with an Oscar for her role in Fences and another standout
              performance in The Help.
        infer:
          steps:
            - deriveFrom: text
              additionalInstructions: >-
                If you find a numeric field use only the value and omit the
                rest.
              type: table
  responses:
    Unauthorized:
      description: >-
        Unauthorized. This error occurs if the authentication token is missing
        or invalid.
      content:
        application/json:
          schema:
            $ref: '#/components/schemas/ErrorResponse'
  securitySchemes:
    bearerAuth:
      type: http
      description: >
        Use a Bearer token for authentication. Submit the token using the
        `Authorization`

        header: `Authorization: Bearer <token>`.
      scheme: bearer
    openId:
      type: oauth2
      flows:
        implicit:
          authorizationUrl: >-
            https://login.datalinks.com/realms/datalinks-realm/protocol/openid-connect/auth
          scopes:
            openid: openid

````