LocalFile
dorsal.file.dorsal_file.LocalFile
LocalFile(
file_path,
client=None,
model_runner_pipeline="default",
use_cache=True,
overwrite_cache=False,
offline=False,
_file_record=None,
)
Bases: _DorsalFile
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
file_path
|
str
|
Absolute or relative path to the local file. |
required |
client
|
DorsalClient | None
|
An optional DorsalClient instance to use for |
None
|
model_runner_pipeline
|
str | list[dict[str, Any]] | None
|
Optional configuration for the ModelRunner instance. |
'default'
|
use_cache
|
bool
|
Whether to use the local cache to speed up processing. Defaults to True. |
True
|
overwrite_cache
|
bool
|
Whether to run the full pipeline and overwrite the cache result. Defaults to False |
False
|
offline
|
bool
|
If True, puts the instance in Offline Mode. Blocks network calls from |
False
|
Raises:
| Type | Description |
|---|---|
FileNotFoundError
|
If the file_path does not exist or is not a file. |
IOError
|
If there are issues reading the file. |
DorsalClientError
|
If model runner encounters an issue that it wraps. |
TypeError
|
If file_path is not a string. |
Source code in venv/lib/python3.13/site-packages/dorsal/file/dorsal_file.py
1153 1154 1155 1156 1157 1158 1159 1160 1161 1162 1163 1164 1165 1166 1167 1168 1169 1170 1171 1172 1173 1174 1175 1176 1177 1178 1179 1180 1181 1182 1183 1184 1185 1186 1187 1188 1189 1190 1191 1192 1193 1194 1195 1196 1197 1198 1199 1200 1201 1202 1203 1204 1205 1206 1207 1208 1209 1210 1211 1212 1213 1214 1215 1216 1217 1218 1219 1220 1221 1222 1223 1224 | |
model
instance-attribute
Represents a file on the local filesystem.
Triggers an offline metadata extraction pipeline that generates/infers metadata for this file. Includes methods for updating, managing and indexing (to DorsalHub) the file metadata.
Attributes:
| Name | Type | Description |
|---|---|---|
hash |
str
|
The primary SHA-256 hash of the file content. |
name |
str
|
The base name of the file. |
size |
int
|
The file size in bytes. |
media_type |
str
|
The detected media type of the file. |
tags |
list[FileTag]
|
A list of tags associated with the file. |
annotations |
object
|
A container for detailed metadata records.
Specific annotations like |
Example
from dorsal import LocalFile
# This line processes the file and populates its metadata.
local_file = LocalFile("path/to/my/document.pdf")
# Strict Offline usage (Blocks all network calls e.g. for validation or indexing)
offline_file = LocalFile("path/to/doc.pdf", offline=True)
print(f"Hashed {local_file.name} ({local_file.size} bytes)")
if local_file.pdf:
print(f"It has {local_file.pdf.page_count} pages.")
add_classification
add_classification(
labels,
*,
vocabulary=None,
source=None,
score_explanation=None,
vocabulary_url=None,
private=True,
overwrite=False,
api_key=None,
ignore_linter_errors=False,
force=False
)
Adds an 'open/classification' annotation to the file.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
- labels
|
can be simple strings (e.g., ["cat"]) or dictionaries |
required |
example
Only labels
lf.add_classification(labels=["EXPIRED", "COMPLETED"])
Labels with vocabulary
lf.add_classification(labels=["eng"], vocabulary=["eng", "fra", "deu"])
Labels, vocabulary, attributes and source
lf.add_classification( labels=[ { "label": "SENSITIVE", "score": 0.95, "attributes": { "page_number": 22, "context": "This document contains sensitive information" } } ], vocabulary=["SENSITIVE", "INTERNAL", "PUBLIC"], source="MySensitiveDocumentScannerV1.0" )
Source code in venv/lib/python3.13/site-packages/dorsal/file/dorsal_file.py
add_embedding
add_embedding(
vector,
*,
model=None,
target=None,
source=None,
private=True,
overwrite=False,
api_key=None,
ignore_linter_errors=False,
force=False
)
Adds an 'open/embedding' annotation to the file.
This helper provides a convenience wrapper for adding a simple embedding (feature vector) to the file record.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
vector
|
list[float]
|
The embedding vector. |
required |
model
|
str
|
Name of the algorithm or model that generated the embedding (e.g., 'CLIP', 'text-embedding-ada-002'). |
None
|
target
|
str
|
Name of target feature/variable |
None
|
source
|
str
|
An optional string describing the source of the annotation (e.g., 'Local CLIP Model v1.2'). This will be passed to the 'detail' field. |
None
|
private
|
bool
|
Whether the annotation should be private. |
True
|
overwrite
|
bool
|
Whether to overwrite an existing annotation. |
False
|
api_key
|
str
|
API key for validation. |
None
|
Returns:
| Type | Description |
|---|---|
'LocalFile'
|
The LocalFile instance, for method chaining. |
Raises:
| Type | Description |
|---|---|
ValueError
|
If the file is missing a 'validation_hash'. |
Source code in venv/lib/python3.13/site-packages/dorsal/file/dorsal_file.py
add_label
Adds a private 'label' tag to the local file model.
This method modifies self.model.tags locally.
To synchronize these tags with DorsalHub, call push on the instance.
Note: This is strictly a private tag because "label" is not a whitelisted public tag namespace on DorsalHub.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
value
|
str | bool | int | float | datetime
|
Value of the label (str, bool, datetime, int, or float). |
required |
api_key
|
str | None
|
Optional API key to use for validation. |
None
|
Source code in venv/lib/python3.13/site-packages/dorsal/file/dorsal_file.py
add_llm_output
add_llm_output(
model,
response_data,
*,
prompt=None,
language=None,
score=None,
score_explanation=None,
generation_params=None,
generation_metadata=None,
source=None,
private=True,
overwrite=False,
api_key=None,
ignore_linter_errors=False,
force=False
)
Adds an 'open/llm-output' annotation to the file.
This helper provides a convenience wrapper for storing the output of a Large Language Model (LLM) task related to this file.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
model
|
str
|
The ID or name of the generative model used (e.g., 'gpt-4o'). |
required |
response_data
|
str | dict
|
The generative output from the model. Can be a string or a simple key-value dictionary. |
required |
prompt
|
str
|
The text-based task or prompt provided to the model. |
None
|
language
|
str
|
The 3-letter ISO-639-3 language code of the response (e.g., 'eng'). |
None
|
score
|
float
|
An optional confidence or evaluation score for the generated output, from -1 to 1. |
None
|
generation_params
|
dict
|
Optional parameters sent in the API request (e.g., {"temperature": 0.5, "max_tokens": 1000}). |
None
|
generation_metadata
|
dict
|
Optional metadata returned by the API response (e.g., {"usage": {...}, "finish_reason": "stop"}). |
None
|
source
|
str
|
An optional string describing the source of the annotation (e.g., 'OpenAI Summarizer v3'). This will be passed to the 'detail' field. |
None
|
private
|
bool
|
Whether the annotation should be private. |
True
|
overwrite
|
bool
|
Whether to overwrite an existing annotation. |
False
|
api_key
|
str
|
API key for validation. |
None
|
Returns:
| Type | Description |
|---|---|
'LocalFile'
|
The LocalFile instance, for method chaining. |
Raises:
| Type | Description |
|---|---|
ValueError
|
If the file is missing a 'validation_hash'. |
Source code in venv/lib/python3.13/site-packages/dorsal/file/dorsal_file.py
2253 2254 2255 2256 2257 2258 2259 2260 2261 2262 2263 2264 2265 2266 2267 2268 2269 2270 2271 2272 2273 2274 2275 2276 2277 2278 2279 2280 2281 2282 2283 2284 2285 2286 2287 2288 2289 2290 2291 2292 2293 2294 2295 2296 2297 2298 2299 2300 2301 2302 2303 2304 2305 2306 2307 2308 2309 2310 2311 2312 2313 2314 2315 2316 2317 2318 2319 2320 2321 2322 2323 2324 2325 2326 2327 2328 2329 2330 2331 2332 2333 | |
add_location
add_location(
longitude,
latitude,
*,
timestamp=None,
camera_make=None,
camera_model=None,
bbox=None,
source=None,
private=True,
overwrite=False,
api_key=None,
ignore_linter_errors=False,
force=False
)
Adds an 'open/geolocation' annotation for a simple Point.
This helper provides a convenience wrapper for the common use case of tagging a file with a single GPS coordinate (longitude, latitude) and optional EXIF-like data.
It automatically builds the required GeoJSON Feature object.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
longitude
|
float
|
The longitude coordinate (e.g., -0.5895). |
required |
latitude
|
float
|
The latitude coordinate (e.g., 51.3814). |
required |
timestamp
|
str
|
An ISO 8601 timestamp for when the geospatial data was captured (e.g., "2025-09-17T11:45:00Z"). |
None
|
camera_make
|
str
|
The make of the camera or sensor. |
None
|
camera_model
|
str
|
The model of the camera or sensor. |
None
|
source
|
str
|
An optional string describing the source of the annotation (e.g., 'EXIF Data Parser'). This will be passed to the 'detail' field. |
None
|
private
|
bool
|
Whether the annotation should be private. |
True
|
overwrite
|
bool
|
Whether to overwrite an existing annotation. |
False
|
api_key
|
str
|
API key for validation. |
None
|
Returns:
| Type | Description |
|---|---|
'LocalFile'
|
The LocalFile instance, for method chaining. |
Raises:
| Type | Description |
|---|---|
ValueError
|
If the file is missing a 'validation_hash'. |
Source code in venv/lib/python3.13/site-packages/dorsal/file/dorsal_file.py
2335 2336 2337 2338 2339 2340 2341 2342 2343 2344 2345 2346 2347 2348 2349 2350 2351 2352 2353 2354 2355 2356 2357 2358 2359 2360 2361 2362 2363 2364 2365 2366 2367 2368 2369 2370 2371 2372 2373 2374 2375 2376 2377 2378 2379 2380 2381 2382 2383 2384 2385 2386 2387 2388 2389 2390 2391 2392 2393 2394 2395 2396 2397 2398 2399 2400 2401 2402 2403 2404 2405 2406 2407 | |
add_private_annotation
add_private_annotation(
*,
schema_id,
annotation_record,
validator=None,
source=None,
api_key=None,
overwrite=False,
ignore_linter_errors=False,
force=False
)
Adds a private annotation to the local file model.
This is a wrapper for the _add_annotation method,
pre-setting private=True.
The annotation is added locally and will be synchronized with DorsalHub upon calling push().
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
schema_id
|
str
|
The schema used for validation (e.g., 'open/generic'). |
required |
annotation_record
|
BaseModel | dict[str, Any]
|
The annotation data (a Pydantic model or dict). |
required |
validator
|
BaseModel | None
|
An optional Pydantic model class or |
None
|
source
|
str | None
|
An optional string describing the source of the manual data. |
None
|
api_key
|
str | None
|
An optional API key for fetching the schema. |
None
|
overwrite
|
bool
|
If True, overwrite an existing annotation for the same dataset. |
False
|
Returns:
| Type | Description |
|---|---|
'LocalFile'
|
The LocalFile instance, for method chaining. |
Raises:
| Type | Description |
|---|---|
ValueError
|
If the |
FileAnnotatorError
|
If the annotation record cannot be processed. |
Example
Source code in venv/lib/python3.13/site-packages/dorsal/file/dorsal_file.py
2009 2010 2011 2012 2013 2014 2015 2016 2017 2018 2019 2020 2021 2022 2023 2024 2025 2026 2027 2028 2029 2030 2031 2032 2033 2034 2035 2036 2037 2038 2039 2040 2041 2042 2043 2044 2045 2046 2047 2048 2049 2050 2051 2052 2053 2054 2055 2056 2057 2058 2059 2060 2061 2062 2063 2064 2065 2066 2067 2068 2069 | |
add_private_tag
Adds a private file tag to the local file model.
This method modifies self.model.tags locally.
To synchronize these tags with DorsalHub, call push on the instance.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
name
|
str
|
Name of the tag (typically 3-64 alphanumeric characters and underscores |
required |
value
|
str | bool | int | float | datetime
|
Value of the tag (str, bool, datetime, int, or float). |
required |
api_key
|
str | None
|
Optional API key to use for validation |
None
|
Source code in venv/lib/python3.13/site-packages/dorsal/file/dorsal_file.py
add_public_annotation
add_public_annotation(
*,
schema_id,
annotation_record,
validator=None,
source=None,
api_key=None,
overwrite=False,
ignore_linter_errors=False,
force=False
)
Adds a public annotation to the local file model.
This is a wrapper for the _add_annotation method,
pre-setting private=False.
The annotation is added locally and will be synchronized with DorsalHub upon calling push().
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
schema_id
|
str
|
The schema used for validation (e.g., 'open/generic'). |
required |
annotation_record
|
BaseModel | dict[str, Any]
|
The annotation data (a Pydantic model or dict). |
required |
validator
|
BaseModel | None
|
An optional Pydantic model class or |
None
|
source
|
str | None
|
An optional string describing the source of the manual data. |
None
|
api_key
|
str | None
|
An optional API key for fetching the schema. |
None
|
overwrite
|
bool
|
If True, overwrite an existing annotation for the same dataset. |
False
|
Returns:
| Type | Description |
|---|---|
'LocalFile'
|
The LocalFile instance, for method chaining. |
Raises:
| Type | Description |
|---|---|
ValueError
|
If the |
FileAnnotatorError
|
If the annotation record cannot be processed. |
Example
Source code in venv/lib/python3.13/site-packages/dorsal/file/dorsal_file.py
2071 2072 2073 2074 2075 2076 2077 2078 2079 2080 2081 2082 2083 2084 2085 2086 2087 2088 2089 2090 2091 2092 2093 2094 2095 2096 2097 2098 2099 2100 2101 2102 2103 2104 2105 2106 2107 2108 2109 2110 2111 2112 2113 2114 2115 2116 2117 2118 2119 2120 2121 2122 2123 2124 2125 2126 2127 2128 2129 2130 2131 | |
add_public_tag
Adds a public file tag to the local file model.
This method modifies self.model.tags locally.
To synchronize these tags with DorsalHub, call push on the instance.
When auto_validate is True, validates the tag against the API
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
name
|
str
|
Name of the tag (typically 3-64 alphanumeric characters and underscores, subject to server-side validation if dorsal is online). |
required |
value
|
str | bool | int | float | datetime
|
Value of the tag (str, bool, datetime, int, or float). |
required |
api_key
|
str | None
|
Optional API key to use for validation |
None
|
Source code in venv/lib/python3.13/site-packages/dorsal/file/dorsal_file.py
add_regression
add_regression(
value,
*,
target=None,
unit=None,
producer=None,
score_explanation=None,
statistic=None,
quantile_level=None,
interval_lower=None,
interval_upper=None,
score=None,
timestamp=None,
attributes=None,
source=None,
private=True,
overwrite=False,
api_key=None,
ignore_linter_errors=False,
force=False
)
Adds an 'open/regression' annotation for a single point estimate.
This helper creates a record containing a single data point.
Use for scalar predictions (e.g. a price).
For multi-point data (e.g. time-series, distributions), use add_private_annotation / add_public_annotation and
construct the record manually with a list of points using dorsal.file.helpers.build_regression_point
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
value
|
float | None
|
The predicted or sampled value. |
required |
target
|
str
|
The name of the variable being predicted (e.g., 'house_price', 'credit_score'). |
None
|
unit
|
str
|
The unit of measurement (e.g., 'USD', 'kg'). |
None
|
statistic
|
str
|
The statistical nature of this value (e.g., 'mean', 'median', 'max', 'quantile'). |
None
|
quantile_level
|
float
|
If statistic='quantile', this defines the level (e.g., 0.95). |
None
|
interval_lower
|
float
|
The lower bound of the confidence interval. |
None
|
interval_upper
|
float
|
The upper bound of the confidence interval. |
None
|
timestamp
|
str | datetime
|
The specific time this prediction applies to. |
None
|
source
|
str
|
An optional string describing the source of the annotation (e.g., 'PricePredictor v1.0'). |
None
|
private
|
bool
|
Whether the annotation should be private. |
True
|
overwrite
|
bool
|
Whether to overwrite an existing annotation. |
False
|
api_key
|
str
|
API key for validation. |
None
|
Returns:
| Type | Description |
|---|---|
'LocalFile'
|
The LocalFile instance, for method chaining. |
Examples:
Simple Point Estimate:
Prediction with Confidence Interval:
>>> lf.add_regression(
... target="temperature",
... value=22.5,
... unit="celsius",
... interval_lower=21.0,
... interval_upper=24.0
... )
Quantile Prediction:
>>> lf.add_regression(
... target="latency",
... value=150,
... unit="ms",
... statistic="quantile",
... quantile_level=0.99
... )
Source code in venv/lib/python3.13/site-packages/dorsal/file/dorsal_file.py
2477 2478 2479 2480 2481 2482 2483 2484 2485 2486 2487 2488 2489 2490 2491 2492 2493 2494 2495 2496 2497 2498 2499 2500 2501 2502 2503 2504 2505 2506 2507 2508 2509 2510 2511 2512 2513 2514 2515 2516 2517 2518 2519 2520 2521 2522 2523 2524 2525 2526 2527 2528 2529 2530 2531 2532 2533 2534 2535 2536 2537 2538 2539 2540 2541 2542 2543 2544 2545 2546 2547 2548 2549 2550 2551 2552 2553 2554 2555 2556 2557 2558 2559 2560 2561 2562 2563 2564 2565 2566 2567 2568 2569 2570 2571 2572 2573 2574 2575 2576 2577 2578 2579 2580 2581 2582 2583 2584 2585 | |
add_transcription
add_transcription(
text,
language,
*,
track_id=None,
source=None,
private=True,
overwrite=False,
api_key=None,
ignore_linter_errors=False,
force=False
)
Adds a simple 'open/audio-transcription' annotation to the file.
This helper provides a convenience wrapper for the common use case of storing the full, flat text transcription of an audio file.
NOTE: This helper populates the top-level 'text' field. It does NOT handle 'segments'. For timed transcriptions, build the dictionary and use the generic 'add_private_annotation' method.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
text
|
str
|
The full, concatenated transcribed text. |
required |
language
|
str
|
The 3-letter ISO-639-3 language code of the transcription (e.g., 'eng'). |
required |
track_id
|
str | int
|
Identifier for the specific audio track or channel in the source file. |
None
|
source
|
str
|
An optional string describing the source of the annotation (e.g., 'Whisper v3 (simple)'). This will be passed to the 'detail' field. |
None
|
private
|
bool
|
Whether the annotation should be private. |
True
|
overwrite
|
bool
|
Whether to overwrite an existing annotation. |
False
|
api_key
|
str
|
API key for validation. |
None
|
Returns:
| Type | Description |
|---|---|
'LocalFile'
|
The LocalFile instance, for method chaining. |
Raises:
| Type | Description |
|---|---|
ValueError
|
If the file is missing a 'validation_hash'. |
Source code in venv/lib/python3.13/site-packages/dorsal/file/dorsal_file.py
2409 2410 2411 2412 2413 2414 2415 2416 2417 2418 2419 2420 2421 2422 2423 2424 2425 2426 2427 2428 2429 2430 2431 2432 2433 2434 2435 2436 2437 2438 2439 2440 2441 2442 2443 2444 2445 2446 2447 2448 2449 2450 2451 2452 2453 2454 2455 2456 2457 2458 2459 2460 2461 2462 2463 2464 2465 2466 2467 2468 2469 2470 2471 2472 2473 2474 2475 | |
from_json
classmethod
Factory method: Instantiates a LocalFile from a JSON File Record.
Source code in venv/lib/python3.13/site-packages/dorsal/file/dorsal_file.py
get_annotations
Retrieves a list of annotations from the local model by schema_id.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
schema_id
|
str
|
The unique identifier of the dataset/schema. |
required |
source_id
|
str | None
|
Optional. Filter custom annotations by their source ID. |
None
|
Returns:
| Type | Description |
|---|---|
Sequence[Annotation | AnnotationXL | PDFValidationModel | MediaInfoValidationModel | EbookValidationModel | OfficeDocumentValidationModel]
|
A list of annotation objects (Core models or generic Annotations). |
Source code in venv/lib/python3.13/site-packages/dorsal/file/dorsal_file.py
1892 1893 1894 1895 1896 1897 1898 1899 1900 1901 1902 1903 1904 1905 1906 1907 1908 1909 1910 1911 1912 1913 1914 1915 1916 1917 1918 1919 1920 1921 1922 1923 1924 1925 1926 1927 1928 1929 1930 1931 1932 1933 1934 1935 1936 1937 1938 1939 1940 1941 1942 1943 1944 1945 1946 1947 1948 1949 1950 1951 1952 1953 1954 | |
push
Indexes file's metadata (annotations and tags) to DorsalHub.
If no record exists for this hash, a new record is created either privately or publicly.
When private is True, if you indexed any annotations or tags for this file before, they are overwritten.
When private is False, the record is updated in an aggregate manner.
Note:
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
private
|
bool
|
If True, the file record will be created as private and will only be accessible to the authenticated user. Defaults to True. |
True
|
api_key
|
str
|
An API key to use for this specific request, overriding the client's default key. Defaults to None. |
None
|
Returns:
| Name | Type | Description |
|---|---|---|
FileIndexResponse |
FileIndexResponse
|
A response object from the API detailing the result of the indexing operation. |
Raises:
| Type | Description |
|---|---|
DorsalClientError
|
If the push operation fails due to an API error, network issue, or authentication failure. |
Source code in venv/lib/python3.13/site-packages/dorsal/file/dorsal_file.py
1504 1505 1506 1507 1508 1509 1510 1511 1512 1513 1514 1515 1516 1517 1518 1519 1520 1521 1522 1523 1524 1525 1526 1527 1528 1529 1530 1531 1532 1533 1534 1535 1536 1537 1538 1539 1540 1541 1542 1543 1544 1545 1546 1547 1548 1549 1550 1551 1552 1553 1554 1555 1556 1557 1558 1559 1560 1561 1562 1563 1564 1565 1566 1567 1568 1569 1570 1571 1572 | |
remove_annotation
Removes an annotation from the local file model.
For custom schemas, providing source_id removes only that specific entry.
Source code in venv/lib/python3.13/site-packages/dorsal/file/dorsal_file.py
save
Exports the File Record to a JSON file on disk.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
path
|
str | Path
|
The file path to write to. |
required |
indent
|
int | None
|
JSON indentation level. |
2
|
by_alias
|
bool
|
Whether to use field aliases (required for correct schema loading). |
True
|
exclude_none
|
bool
|
Whether to exclude fields with None values. |
True
|
Raises:
| Type | Description |
|---|---|
IOError
|
If the file cannot be written. |
Source code in venv/lib/python3.13/site-packages/dorsal/file/dorsal_file.py
to_dict
Overrides the parent method to include local file information.
Source code in venv/lib/python3.13/site-packages/dorsal/file/dorsal_file.py
to_json
Export the File Record to a JSON string.
Source code in venv/lib/python3.13/site-packages/dorsal/file/dorsal_file.py
validate_tags
Validates all tags against DorsalHub's API.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
api_key
|
str | None
|
Optional API key. |
None
|
Returns:
| Type | Description |
|---|---|
|
The validation response object from the client. |
Raises:
| Type | Description |
|---|---|
DorsalError
|
If the instance is in offline mode. |
InvalidTagError
|
If the tags are rejected by the API. |
DorsalClientError
|
If the API call fails. |