🍃 Tellia / 🥷🏿 Engineering

Appearance

Main chat weather (Open-Meteo) and semantic search result IDs ​

This document summarizes work on field-scoped weather in the main streaming chat, resolving the correct database IDs for Fields vs ParseResults, and fixing semantic search so downstream tools receive ParseResult row IDs instead of embedding chunk IDs.

1. Goals ​

  • Give the main chat agent (ReAct orchestrator in streaming-chat) a get_weather tool that calls Open-Meteo for forecast data (current, hourly, daily, soil-oriented variables).
  • Tie weather to agricultural fields: resolve coordinates from Field geometry or centroid, scoped to the user’s company.
  • Support agronomic-style questions (spray windows, frost, weekly outlook, soil-related series) via a focus parameter that maps to different Open-Meteo query shapes.
  • Fix failures where the model passed vector embedding document IDs to tools that expect ParseResult or Field document IDs.

2. Open-Meteo integration (apps/agri-backend/src/weather/) ​

2.1 Module layout ​

FileRole
weather.module.tsNest module; provides and exports OpenMeteoClient.
open-meteo.client.tsHTTP fetch to Open-Meteo Forecast API; Zod validation of JSON; builds a compact summary (highlights) for the LLM instead of dumping full hourly arrays.
field-coordinates.util.tsDerives latitude / longitude from a Field document: prefers GeoJSON centroid ([lon, lat]); otherwise averages the exterior ring of Polygon / first polygon of MultiPolygon.
weather.enabled.tsisOpenMeteoEnabled() — reads OPEN_METEO_ENABLED; tool registration is skipped when set to false.

2.2 Environment variables ​

VariablePurpose
OPEN_METEO_ENABLEDIf set to false, get_weather is not registered in main chat (getTools / getToolHandlers). Default behavior: enabled when unset.
OPEN_METEO_BASE_URLOptional override for the forecast API base URL (defaults to https://api.open-meteo.com/v1/forecast). Useful for tests or proxies.

No API key is required for the public Open-Meteo endpoint; refer to Open-Meteo terms and pricing for commercial / high-volume use.

2.3 focus modes (client) ​

The client maps focus to different current / hourly / daily parameter sets, for example:

  • general — short overview (current + hourly + some daily).
  • spray_window — wind, precipitation probability, humidity, etc., over a few days.
  • frost — hourly temperature emphasis; summary includes hints such as min temp in next 24h and hours below thresholds.
  • weekly — daily max/min temperature, precipitation, wind max, weather code for ~7 days.
  • soil — hourly soil temperature / moisture variables where the API exposes them (e.g. soil_temperature_6cm, soil_moisture_3_9cm).

3. Main chat wiring ​

3.1 StreamingChatModule ​

  • Imports FieldModule so MainChatAgentService can inject FieldsService.
  • Imports WeatherModule for OpenMeteoClient.

File: apps/agri-backend/src/streaming-chat/streaming-chat.module.ts.

3.2 MainChatAgentService ​

File: apps/agri-backend/src/streaming-chat/services/main-chat-agent.service.ts.

Tool: get_weather (when Open-Meteo is enabled)

  • Parameters

    • field_id (required): Identifier the agent obtained from read_data for the field row or field record.
    • focus (optional): One of general, spray_window, frost, weekly, soil.

  • Handler flow

    1. Resolve the Field document via FieldsService.findFieldDocumentForWeather (see §4).
    2. Compute lat/lon with resolveFieldLatLon.
    3. Call OpenMeteoClient.fetchForecastSummary.
    4. Return a structured payload (success, field_name, location, forecast, etc.) for the ReAct loop.

System prompt updates describe:

  • The third tool alongside read_data and write_data.
  • That weather is forecast / model data, not a substitute for a licensed PCA for spray decisions (aligned with existing guardrails).
  • Workflow: resolve field context with read_data when needed, then get_weather.

4. Field resolution: findFieldDocumentForWeather ​

4.1 Problem ​

The UI and read_data / ReportGenerationAgent often surface IDs that are not the MongoDB _id of the fields collection:

  • Fields workspace rows live in parseresults (ParseResult documents) with a field property pointing at the real Field document.
  • The agent might pass the ParseResult _id (table row) or, before the search fix, wrong IDs entirely (see §5).

Calling findByIdWithAccess only on fields therefore returned 404 for common IDs.

4.2 Solution ​

FieldsService.findFieldDocumentForWeather(id, user) in apps/agri-backend/src/fields/fields.service.ts resolves in order:

  1. Field by _id (with company / sharing filter via applyCompanyFilter).
  2. ParseResult by _id; if present, read field (24-hex ObjectId string); load Field by that id.
  3. Field by internalReferenceId (when the id is the stable reference id).
  4. ParseResult by internalReferenceId; again follow field to Field.

If nothing matches, it throws the same not-found pattern as other company-scoped resources (I18nExceptions.notFound).

This mirrors the idea already used in updateFieldWithParseResult, where the API accepts either a Field id or a ParseResult id and follows parseResult.field.

4.3 Coordinates ​

If the Field has no usable centroid or geometry, resolveFieldLatLon throws a clear error (user must fix boundaries in the app).

5. Semantic search: id vs embedding chunk (Report Generation Agent) ​

5.1 Problem ​

search_parse_results runs a vector search on parseResultEmbeddingsModel. Each hit is an embedding chunk document whose _id is not the ParseResult workspace row id.

The tool previously set:

ts
id: doc._id?.toString() || doc.parseResultId?.toString();

So results[].id was almost always the chunk id. The main chat model then passed that value to get_weather as field_id, which findFieldDocumentForWeather could not resolve → “field not found / access” style failures.

5.2 Fix (backend shaping) ​

File: apps/agri-backend/src/agents/report-generation-agent/tools/search-parse-results.tool.ts.

  • id is now parseResultId first, falling back to _id only if parseResultId is missing.
  • embeddingChunkId is set when the chunk _id differs from id, so the model and logs can distinguish row id vs chunk id.

Tool description text states explicitly:

  • Use id as the ParseResult row id (same logical record as query_parse_results _id).
  • Do not use embeddingChunkId for tools that need ParseResult / Field ids (e.g. get_weather).

5.3 Report agent system prompt ​

File: apps/agri-backend/src/agents/report-generation-agent/utils/system-prompt.utils.ts.

Under the semantic search strategy, a line was added: each result’s id is the ParseResult row id, not the embedding chunk id.

5.4 Remaining caveat ​

Semantic search can still return a chunk whose parseResultId belongs to another row (irrelevant hit). id is then a valid ParseResult id but possibly the wrong field. For exact field rows, query_parse_results on the Fields workspace remains the most reliable path; the search fix ensures the type of id is correct when the hit is relevant.

6. End-to-end flow (successful path) ​

7. Files touched (reference) ​

AreaPath
Weatherapps/agri-backend/src/weather/*
Streaming chat moduleapps/agri-backend/src/streaming-chat/streaming-chat.module.ts
Main chat agentapps/agri-backend/src/streaming-chat/services/main-chat-agent.service.ts
Field resolutionapps/agri-backend/src/fields/fields.service.ts
Semantic search toolapps/agri-backend/src/agents/report-generation-agent/tools/search-parse-results.tool.ts
Report agent promptapps/agri-backend/src/agents/report-generation-agent/utils/system-prompt.utils.ts
Weather integration testsapps/agri-backend/src/weather/weather.integration.spec.ts

8. Verification ​

  • Typecheck: pnpm --filter agri-backend exec tsc --noEmit
  • Integration tests (require test DB/containers, e.g. pnpm --filter agri-backend test:up then pnpm exec jest weather.integration.spec.ts from apps/agri-backend): src/weather/weather.integration.spec.ts covers OpenMeteoClient (mocked fetch), findFieldDocumentForWeather, and get_weather via MainChatAgentService.getToolHandlers().
  • Manual: Ask main chat for weather for a named field after read_data returns a Fields row; confirm get_weather succeeds and coordinates match the field map.
  • After the search fix: if the agent uses search_parse_results ids for follow-up, those id values should be parseResultId-based and resolvable when the row links to a field.

9. Related documentation ​