/v1/schemas/gemini/video/gemini-omni-1.2-flash
| field | recorded source |
|---|---|
| schema | Source linked above. generated Source tracea85d974f98e66ba177124a84492b44e557dac22fdf35f545c0b9c8ef8803b351{
"fetchedAt": 1795152603,
"extractorVersion": "2",
"verifiedAt": null
} |
Request body
23 properties · 2 required · 88 $defs| property | type | description | constraints |
|---|---|---|---|
| model* | enum | The model that generates the video. | gemini-omni-1.2-flash |
| input* | AudioContent | DocumentContent | ImageContent | Step[] | Content[] | … | The input for the interaction. | |
| generation_config | GenerationConfig | Input only. Configuration parameters for the model interaction. | |
| id | string | Required. Output only. A unique identifier for the interaction completion. | |
| status | enum | Required. Output only. The status of the interaction. | in_progress | requires_action | completed | failed | cancelled | incomplete · +2 more |
| created | string | Required. Output only. The time at which the response was created in ISO 8601 format (YYYY-MM-DDThh:mm:ssZ). | |
| updated | string | Required. Output only. The time at which the response was last updated in ISO 8601 format (YYYY-MM-DDThh:mm:ssZ). | |
| system_instruction | string | System instruction for the interaction. | |
| tools | Tool[] | A list of tool declarations the model may call during interaction. | |
| response_modalities | ResponseModality[] | The requested modalities of the response (TEXT, IMAGE, AUDIO). | deprecated |
| response_format | AudioResponseFormat | ImageResponseFormat | TextResponseFormat | ResponseFormat[] | Enforces that the generated response is a JSON object that complies with the JSON schema specified in this field. | |
| response_mime_type | string | The mime type of the response. This is required if response_format is set. | deprecated |
| previous_interaction_id | string | The ID of the previous interaction, if any. | |
| environment | object | string | The environment configuration for the interaction. Can be an object specifying remote environment sources or a string referencing an existing environment ID. | |
| environment_id | string | Output only. The environment ID for the interaction. Only populated if environment config is set in the request. | |
| service_tier | enum | The service tier for the interaction. | flex | standard | priority |
| webhook_config | WebhookConfig | Optional. Webhook configuration for receiving notifications when the interaction completes. | |
| safety_settings | SafetySetting[] | Safety settings for the interaction. | |
| labels | object | The labels with user-defined metadata for the request. | |
| continuation_token | string | Opaque token to resume a long decode. Output: set when status is REQUIRES_CONTINUATION and decoding can be resumed. Input: pass the latest token back unchanged in CreateInteraction to continue decoding. | format: byte |
| stream | boolean | Input only. Whether the interaction will be streamed. | |
| store | boolean | Input only. Whether to store the response and request for later retrieval. | |
| background | boolean | Input only. Whether to run the model interaction in the background. |
Types
88 $defsAllowedToolsobject
The configuration for allowed tools.
| mode | enum | The mode of the tool choice. | auto | any | none | validated |
| tools | string[] | The names of the allowed tools. |
Annotationobject | object | object | …
Citation information for model-generated content.
AudioContentobject
An audio content block.
| type* | const | "audio" | |
| data | string | The audio content. | format: byte |
| uri | string | The URI of the audio. | |
| channels | integer | The number of audio channels. | format: int32 |
| sample_rate | integer | The sample rate of the audio. | format: int32 |
| processing | AudioStaticMediaProcessing | enum | How the model processes this audio for understanding. | |
| mime_type | enum | The mime type of the audio. | audio/wav | audio/mp3 | audio/aiff | audio/aac | audio/ogg | audio/flac · +7 more |
AudioMediaProcessingAudioStaticMediaProcessing
| type* | const | "static" | |
| start_offset | string | Optional. Segment start time, relative to the start of the audio stream. Specified as a decimal number of seconds followed by an 's' suffix, e.g., "10.5s". Must be non-negative. | format: google-duration |
| end_offset | string | Optional. Segment end time, relative to the start of the audio stream. Specified as a decimal number of seconds followed by an 's' suffix, e.g., "30s". Must be positive and greater than `start_offset` if `start_offset` is set. | format: google-duration |
AudioResponseFormatobject
Configuration for audio output format.
| type* | const | "audio" | |
| mime_type | enum | The MIME type of the audio output. | audio/mp3 | audio/ogg_opus | audio/l16 | audio/wav | audio/alaw | audio/mulaw |
| delivery | enum | The delivery mode for the audio output. | inline | uri |
| sample_rate | integer | Sample rate in Hz. | format: int32 |
| bit_rate | integer | Bit rate in bits per second (bps). Only applicable for compressed formats (MP3, Opus). | format: int32 |
AudioStaticMediaProcessingobject
| type* | const | "static" | |
| start_offset | string | Optional. Segment start time, relative to the start of the audio stream. Specified as a decimal number of seconds followed by an 's' suffix, e.g., "10.5s". Must be non-negative. | format: google-duration |
| end_offset | string | Optional. Segment end time, relative to the start of the audio stream. Specified as a decimal number of seconds followed by an 's' suffix, e.g., "30s". Must be positive and greater than `start_offset` if `start_offset` is set. | format: google-duration |
CodeExecutionobject
A tool that can be used by the model to execute code.
| type* | const | "code_execution" |
CodeExecutionCallStepobject
Code execution call step.
| arguments* | CodeExecutionCallStepArguments | Required. The arguments to pass to the code execution. | |
| id* | string | Required. A unique ID for this specific tool call. | |
| type* | const | "code_execution_call" | |
| signature | string | A signature hash for backend validation. | format: byte |
CodeExecutionCallStepArgumentsobject
The arguments to pass to the code execution.
| language | enum | Programming language of the `code`. | python | bash |
| code | string | The code to be executed. |
CodeExecutionResultStepobject
Code execution result step.
| result* | Content[] | string | The output of the code execution. | |
| call_id* | string | Required. ID to match the ID from the function call block. | |
| type* | const | "code_execution_result" | |
| is_error | boolean | Whether the code execution resulted in an error. | |
| signature | string | A signature hash for backend validation. | format: byte |
ComputerUseobject
A tool that can be used by the model to interact with the computer.
| type* | const | "computer_use" | |
| environment | enum | The environment being operated. | browser | mobile | desktop |
| excluded_predefined_functions | string[] | The list of predefined functions that are excluded from the model call. | |
| enable_prompt_injection_detection | boolean | Whether enable the prompt injection detection check on computer-use request. | |
| disabled_safety_policies | enum[] | Optional. Disabled safety policies for computer use. |
Contentobject | object | object | …
The content of the response.
DocumentContentobject
A document content block.
| type* | const | "document" | |
| data | string | The document content. | format: byte |
| uri | string | The URI of the document. | |
| resolution | enum | The resolution of the media. Applicable when PAGE_IMAGE is selected in content_layers. | low | medium | high | ultra_high |
| processing | DocumentStaticMediaProcessing | enum | How the model processes this document for understanding. | |
| mime_type | enum | The mime type of the document. | application/pdf | text/csv | application/vnd.openxmlformats-… | application/vnd.ms-powerpoint | application/vnd.oasis.opendocum… | application/vnd.openxmlformats-… · +6 more |
DocumentMediaProcessingDocumentStaticMediaProcessing
| type* | const | "static" | |
| page_options | PageProcessingOptions | Options for paginated documents. Only valid for paginated document MIME types (e.g. PDF, Office documents, and presentations). | |
| spreadsheet_options | SpreadsheetProcessingOptions | Options for spreadsheets. Only valid for spreadsheet MIME types (.xlsx, .xls, .ods). |
DocumentStaticMediaProcessingobject
| type* | const | "static" | |
| page_options | PageProcessingOptions | Options for paginated documents. Only valid for paginated document MIME types (e.g. PDF, Office documents, and presentations). | |
| spreadsheet_options | SpreadsheetProcessingOptions | Options for spreadsheets. Only valid for spreadsheet MIME types (.xlsx, .xls, .ods). |
EgressRuleobject
A single domain allowlist rule with optional header injection.
| domain* | string | Domain to allow outbound requests to. Supports wildcards (e.g. '*.googleapis.com'). Use '*' to allow all domains. | |
| transform | object | Headers to inject into requests matching this rule. Key: header name (e.g., "Authorization"). Value: header value (e.g., "Bearer your-token"). | |
| credential | string | Optional. Reference to a server-managed Credential resource by ID. |
EnvironmentConfigobject
Configuration for a custom environment.
| type* | const | "remote" | |
| sources | Source[] | ||
| network | object | enum | Network configuration for the environment. | |
| environment_id | string | Optional. The environment ID for the interaction. If specified, the request will update the existing environment instead of creating a new one. | |
| env | object | string | Environment variables to set in the sandbox environment. | |
| mode | enum | The compute lifecycle mode for this environment. | on_demand | self_hosted |
| custom_config | object | Optional. Arbitrary customer-defined JSON configuration passed through to the self-hosted environment poller. Only supported when `mode` is `SELF_HOSTED`. |
EnvironmentNetworkEgressAllowlistobject
Network egress configuration for the environment.
| allowlist | EgressRule[] | enum | List of allowed domains and their configurations. Set to `"disabled"` to block all network egress. |
EnvVarobject
An environment variable to set in the execution environment.
| value | string | Direct string value for plain environment variables. | |
| credential | string | Optional reference to a server-managed Credential resource by ID. |
FileCitationobject
A file citation annotation.
| type* | const | "file_citation" | |
| document_uri | string | The URI of the file. | |
| file_name | string | The name of the file. | |
| source | string | Source attributed for a portion of the text. | |
| custom_metadata | object | User provided metadata about the retrieved context. | |
| page_number | integer | Page number of the cited document, if applicable. | format: int32 |
| media_id | string | Media ID in-case of image citations, if applicable. | |
| start_index | integer | Start of segment of the response that is attributed to this source. | format: int32 |
| end_index | integer | End of the attributed segment, exclusive. | format: int32 |
FileSearchobject
A tool that can be used by the model to search files.
| type* | const | "file_search" | |
| file_search_store_names | string[] | The file search store names to search. | |
| top_k | integer | The number of semantic retrieval chunks to retrieve. | format: int32 |
| metadata_filter | string | Metadata filter to apply to the semantic retrieval documents and chunks. |
FileSearchCallStepobject
File Search call step.
| id* | string | Required. A unique ID for this specific tool call. | |
| type* | const | "file_search_call" | |
| signature | string | A signature hash for backend validation. | format: byte |
FileSearchResultStepobject
File Search result step.
| call_id* | string | Required. ID to match the ID from the function call block. | |
| type* | const | "file_search_result" | |
| signature | string | A signature hash for backend validation. | format: byte |
Functionobject
A tool that can be used by the model.
| type* | const | "function" | |
| name | string | The name of the function. | |
| description | string | A description of the function. | |
| parameters | any | The JSON Schema for the function's parameters. |
FunctionCallStepobject
A function tool call step.
| name* | string | Required. The name of the tool to call. | |
| arguments* | object | Required. The arguments to pass to the function. | |
| id* | string | Required. A unique ID for this specific tool call. | |
| type* | const | "function_call" |
FunctionResultStepobject
Result of a function tool call.
| result* | ImageContent | TextContent[] | object | string | Required. The result of the tool call. | |
| call_id* | string | Required. ID to match the ID from the function call block. | |
| type* | const | "function_result" | |
| name | string | The name of the tool that was called. | |
| is_error | boolean | Whether the tool call resulted in an error. |
GenerationConfigobject
Configuration parameters for model interactions.
| temperature | number | Controls the randomness of the output. | format: float · deprecated |
| top_p | number | The maximum cumulative probability of tokens to consider when sampling. | format: float · deprecated |
| seed | integer | Seed used in decoding for reproducibility. | format: int32 |
| stop_sequences | string[] | A list of character sequences that will stop output interaction. | |
| tool_choice | object | enum | The tool choice configuration. | |
| thinking_level | enum | The level of thought tokens that the model should generate. | minimal | low | medium | high | xhigh |
| thinking_summaries | enum | Whether to include thought summaries in the response. | auto | none |
| max_output_tokens | integer | The maximum number of tokens to include in the response. | format: int32 |
| image_config | ImageConfig | Configuration for image interaction. | deprecated |
| video_config | VideoConfig | Configuration for video generation. | |
| transcription_config | TranscriptionConfig | Optional. Configuration for speech recognition (transcription). If present, ASR is enabled. | |
| structured_speech_config | object | SpeechConfig[] | Speech and multi-speaker configuration. |
GoogleMapsobject
A tool that can be used by the model to call Google Maps.
| type* | const | "google_maps" | |
| enable_widget | boolean | Whether to return a widget context token in the tool call result of the response. | |
| latitude | number | The latitude of the user's location. | format: double |
| longitude | number | The longitude of the user's location. | format: double |
GoogleMapsCallStepobject
Google Maps call step.
| id* | string | Required. A unique ID for this specific tool call. | |
| type* | const | "google_maps_call" | |
| arguments | GoogleMapsCallStepArguments | The arguments to pass to the Google Maps tool. | |
| signature | string | A signature hash for backend validation. | format: byte |
GoogleMapsCallStepArgumentsobject
The arguments to pass to the Google Maps tool.
| queries | string[] | The queries to be executed. |
GoogleMapsResultItemobject
The result of the Google Maps.
| places | GoogleMapsResultPlaces[] | ||
| widget_context_token | string |
GoogleMapsResultPlacesobject
| place_id | string | ||
| name | string | ||
| url | string | ||
| review_snippets | ReviewSnippet[] |
GoogleMapsResultStepobject
Google Maps result step.
| result* | GoogleMapsResultItem[] | ||
| call_id* | string | Required. ID to match the ID from the function call block. | |
| type* | const | "google_maps_result" | |
| signature | string | A signature hash for backend validation. | format: byte |
GoogleSearchobject
A tool that can be used by the model to search Google.
| type* | const | "google_search" | |
| search_types | enum[] | The types of search grounding to enable. |
GoogleSearchCallStepobject
Google Search call step.
| arguments* | GoogleSearchCallStepArguments | Required. The arguments to pass to Google Search. | |
| id* | string | Required. A unique ID for this specific tool call. | |
| type* | const | "google_search_call" | |
| search_type | enum | The type of search grounding enabled. | web_search | image_search |
| signature | string | A signature hash for backend validation. | format: byte |
GoogleSearchCallStepArgumentsobject
The arguments to pass to Google Search.
| queries | string[] | Web search queries for the following-up web search. |
GoogleSearchResultItemobject
The result of the Google Search.
| search_suggestions | string | Web content snippet that can be embedded in a web page or an app webview. |
GoogleSearchResultStepobject
Google Search result step.
| result* | GoogleSearchResultItem[] | Required. The results of the Google Search. | |
| call_id* | string | Required. ID to match the ID from the function call block. | |
| type* | const | "google_search_result" | |
| is_error | boolean | Whether the Google Search resulted in an error. | |
| signature | string | A signature hash for backend validation. | format: byte |
HarmCategoryenum
hate_speech | dangerous_content | harassment | sexually_explicit | jailbreak
ImageConfigobject
The configuration for image interaction.
| aspect_ratio | string | The aspect ratio of the image to generate. Supported aspect ratios: 1:1, 2:3, 3:2, 3:4, 4:3, 9:16, 16:9, 21:9. | |
| image_size | string | Specifies the size of generated images. Supported values are `1K`, `2K`, `4K`. If not specified, the model will use default value `1K`. |
ImageContentobject
An image content block.
| type* | const | "image" | |
| data | string | The image content. | format: byte |
| uri | string | The URI of the image. | |
| resolution | enum | The resolution of the media. | low | medium | high | ultra_high |
| mime_type | enum | The mime type of the image. | image/png | image/jpeg | image/webp | image/heic | image/heif | image/gif · +2 more |
ImageResponseFormatobject
Configuration for image output format.
| type* | const | "image" | |
| mime_type | enum | The MIME type of the image output. | image/jpeg |
| delivery | enum | The delivery mode for the image output. | inline | uri |
| aspect_ratio | enum | The aspect ratio for the image output. | 1:1 | 2:3 | 3:2 | 3:4 | 4:3 | 4:5 · +8 more |
| image_size | enum | The size of the image output. | 512 | 1K | 2K | 4K |
InteractionsInputAudioContent | DocumentContent | ImageContent | Step[] | Content[] | …
The input for the interaction.
McpServerobject
A MCPServer is a server that can be called by the model to perform actions.
| type* | const | "mcp_server" | |
| name | string | The name of the MCPServer. | |
| url | string | The full URL for the MCPServer endpoint. Example: "https://api.example.com/mcp" | |
| headers | object | Optional: Fields for authentication headers, timeouts, etc., if needed. | |
| allowed_tools | AllowedTools[] | The allowed tools. |
McpServerToolCallStepobject
MCPServer tool call step.
| name* | string | Required. The name of the tool which was called. | |
| server_name* | string | Required. The name of the used MCP server. | |
| arguments* | object | Required. The JSON object of arguments for the function. | |
| id* | string | Required. A unique ID for this specific tool call. | |
| type* | const | "mcp_server_tool_call" |
McpServerToolResultStepobject
MCPServer tool result step.
| result* | ImageContent | TextContent[] | object | string | Required. The output from the MCP server call. Can be simple text or rich content. | |
| call_id* | string | Required. ID to match the ID from the function call block. | |
| type* | const | "mcp_server_tool_result" | |
| name | string | Name of the tool which is called for this specific tool call. | |
| server_name | string | The name of the used MCP server. |
MediaProcessingStaticMediaProcessing
| type* | const | "static" | |
| start_offset | string | Optional. Segment start time. Specified as a decimal number of seconds followed by an 's' suffix, e.g., "10.5s". Must be non-negative. | format: google-duration |
| end_offset | string | Optional. Segment end time. Specified as a decimal number of seconds followed by an 's' suffix, e.g., "30s". Must be non-negative and greater than `start_offset` if `start_offset` is set. | format: google-duration |
| fps | number | Optional. Video frame-rate sampling density. | format: double |
| content_layers | enum[] | Optional. Content layers to extract from the video. |
MediaResolutionenum
low | medium | high | ultra_high
ModelOutputStepobject
Output generated by the model.
| type* | const | "model_output" | |
| content | Content[] |
PageProcessingOptionsobject
Processing options for paginated documents, such as PDFs, Office documents, and presentations.
| start_page | integer | Optional. 1-indexed start page, inclusive. | format: int32 |
| end_page | integer | Optional. 1-indexed end page, inclusive. Must be >= `start_page` if `start_page` is set. | format: int32 |
| content_layers | enum[] | Optional. Content layers to extract from a paginated document. |
PlaceCitationobject
A place citation annotation.
| type* | const | "place_citation" | |
| place_id | string | The ID of the place, in `places/{place_id}` format. | |
| name | string | Title of the place. | |
| url | string | URI reference of the place. | |
| review_snippets | ReviewSnippet[] | Snippets of reviews that are used to generate answers about the features of a given place in Google Maps. | |
| start_index | integer | Start of segment of the response that is attributed to this source. | format: int32 |
| end_index | integer | End of the attributed segment, exclusive. | format: int32 |
ProcessingCallStepobject
A server-initiated processing step for media analysis (e.g. video understanding).
| id* | string | Required. A unique ID for this specific tool call. | |
| type* | const | "processing_call" | |
| signature | string | A signature hash for backend validation. | format: byte |
ProcessingResultStepobject
The result of a server-initiated media processing step.
| call_id* | string | Required. ID to match the ID from the function call block. | |
| type* | const | "processing_result" | |
| signature | string | A signature hash for backend validation. | format: byte |
ResponseFormatobject | object | object | …
ResponseModalityenum
text | image | audio | video | document
ReviewSnippetobject
Encapsulates a snippet of a user review that answers a question about the features of a specific place in Google Maps.
| title | string | Title of the review. | |
| url | string | A link that corresponds to the user review on Google Maps. | |
| review_id | string | The ID of the review snippet. |
SafetySettingobject
A safety setting that affects the safety-blocking behavior.
| type* | enum | Required. The type of harm category to be blocked. | hate_speech | dangerous_content | harassment | sexually_explicit | jailbreak |
| threshold* | enum | Required. The threshold for blocking content. If the harm probability exceeds this threshold, the content will be blocked. | block_low_and_above | block_medium_and_above | block_only_high | off |
| method | enum | Optional. The method for blocking content. If not specified, the default behavior is to use the probability score. | severity | probability |
ServiceTierenum
flex | standard | priority
SmartTranscriptionModeobject
Configuration for smart transcription mode.
| type* | const | "smart" | |
| source_text | string | The source text to be edited or rewritten via voice instructions. When provided, the model performs voice-driven editing and outputs the finalized replacement text. |
Sourceobject
A source to be mounted into the environment.
| type | enum | gcs | inline | repository | |
| source | string | The source of the environment. For Cloud Storage, this is the Cloud Storage path. For GitHub, this is the GitHub path. | |
| target | string | Where the source should appear in the environment. | |
| content | string | The inline content if `type` is `INLINE`. | |
| encoding | string | Optional encoding for inline content (e.g. `base64`). |
SpeakerConfigobject
Configuration for multi-speaker and speech generation.
| speakers | SpeechConfig[] | Individual speaker configurations. |
SpeechAnnotationobject
Speech annotation for text content.
| type* | const | "speech_metadata" | |
| speaker | string | The speaker to associate with this turn. | |
| style | string | Style instruction for the speech synthesis. | |
| start_index | integer | Start of segment of the response that is attributed to this source. | format: int32 |
| end_index | integer | End of the attributed segment, exclusive. | format: int32 |
SpeechConfigobject
The configuration for speech interaction.
| voice | string | The voice of the speaker. | |
| language | string | The language of the speech. | |
| speaker | string | The speaker's name, it should match the speaker name given in the prompt. |
SpreadsheetProcessingOptionsobject
Processing options for spreadsheets.
| visible_cells | string | Optional. The range of cells to extract, in A1 notation (e.g. "A1:D10"). If not set, the full occupied area of the sheet is used. | |
| sheet_index | integer | Optional. The 1-indexed index of the sheet to extract, in workbook order. If not set, all sheets are extracted. | format: int32 |
| content_layers | enum[] | Optional. Content layers to extract from a spreadsheet. |
StaticMediaProcessingobject
| type* | const | "static" | |
| start_offset | string | Optional. Segment start time. Specified as a decimal number of seconds followed by an 's' suffix, e.g., "10.5s". Must be non-negative. | format: google-duration |
| end_offset | string | Optional. Segment end time. Specified as a decimal number of seconds followed by an 's' suffix, e.g., "30s". Must be non-negative and greater than `start_offset` if `start_offset` is set. | format: google-duration |
| fps | number | Optional. Video frame-rate sampling density. | format: double |
| content_layers | enum[] | Optional. Content layers to extract from the video. |
Stepobject | object | object | …
A step in the interaction.
TextContentobject
A text content block.
| text* | string | Required. The text content. | |
| type* | const | "text" | |
| annotations | Annotation[] | Citation information for model-generated content. |
TextResponseFormatobject
Configuration for text output format.
| type* | const | "text" | |
| mime_type | enum | The MIME type of the text output. | application/json | text/plain |
| schema | object | The JSON schema that the output should conform to. Only applicable when mime_type is application/json. |
ThinkingLevelenum
minimal | low | medium | high | xhigh
ThinkingSummariesenum
auto | none
ThoughtStepobject
A thought step.
| type* | const | "thought" | |
| signature | string | A signature hash for backend validation. | format: byte |
| summary | Content[] | A summary of the thought. |
Toolobject | object | object | …
A tool that can be used by the model.
ToolChoiceConfigobject
The tool choice configuration containing allowed tools.
| allowed_tools | AllowedTools | The allowed tools. |
TranscriptionConfigobject
Configuration for speech recognition (transcription).
| language_codes | string[] | Optional. BCP-47 language codes providing hints about the languages present in the audio. If omitted or empty, defaults to automatic language detection. | |
| custom_vocabulary | string[] | Optional. A list of custom vocabulary phrases to bias the speech recognition model toward recognizing specific terms. | |
| mode | SmartTranscriptionMode | VerbatimTranscriptionMode | enum | Discriminated transcription mode options or enum. | |
| timestamp_granularities | string[] | Optional. The granularity of timestamps to include in the transcription output. Supported values: "word". If empty, no timestamps are generated. | deprecated |
| diarization_mode | string | Optional. Configures speaker diarization. Supported values: "speaker". | deprecated |
| adaptation_phrases | string[] | Optional. A list of phrases to bias the ASR model towards. | deprecated |
TranscriptionModeobject | object
Configuration for transcription mode.
UrlCitationobject
A URL citation annotation.
| type* | const | "url_citation" | |
| url | string | The URL. | |
| title | string | The title of the URL. | |
| start_index | integer | Start of segment of the response that is attributed to this source. | format: int32 |
| end_index | integer | End of the attributed segment, exclusive. | format: int32 |
UrlContextobject
A tool that can be used by the model to fetch URL context.
| type* | const | "url_context" |
UrlContextCallArgumentsobject
The arguments to pass to the URL context.
| urls | string[] | The URLs to fetch. |
UrlContextCallStepobject
URL context call step.
| arguments* | UrlContextCallArguments | Required. The arguments to pass to the URL context. | |
| id* | string | Required. A unique ID for this specific tool call. | |
| type* | const | "url_context_call" | |
| signature | string | A signature hash for backend validation. | format: byte |
UrlContextResultobject
The result of the URL context.
| url | string | The URL that was fetched. | |
| status | enum | The status of the URL retrieval. | success | error | paywall | unsafe |
UrlContextResultStepobject
URL context result step.
| result* | UrlContextResult[] | Required. The results of the URL context. | |
| call_id* | string | Required. ID to match the ID from the function call block. | |
| type* | const | "url_context_result" | |
| is_error | boolean | Whether the URL context resulted in an error. | |
| signature | string | A signature hash for backend validation. | format: byte |
UserInputStepobject
Input provided by the user.
| type* | const | "user_input" | |
| content | Content[] |
VerbatimTranscriptionModeobject
Configuration for verbatim transcription mode.
| type* | const | "verbatim" | |
| timestamp_granularities | string[] | Optional. The granularity of timestamps to include in the transcription output. Supported values: "word". If empty, no timestamps are generated. | |
| diarization_mode | string | Optional. Configures speaker diarization. Supported values: "speaker". |
VideoConfigobject
Configuration options for video generation.
| task | enum | Optional task mode for video generation. If not specified, the model automatically determines the appropriate mode based on the provided text prompt and input media. | text_to_video | image_to_video | reference_to_video | edit | extend |
VideoContentobject
A video content block.
| type* | const | "video" | |
| data | string | The video content. | format: byte |
| uri | string | The URI of the video. | |
| resolution | enum | The resolution of the media. | low | medium | high | ultra_high |
| processing | StaticMediaProcessing | enum | How the model processes this video for understanding. | |
| name | string | A user-defined name for this content block. Can be referenced by the model in the final response. | |
| mime_type | enum | The mime type of the video. | video/mp4 | video/mpeg | video/mpg | video/mov | video/avi | video/x-flv · +3 more |
VideoResponseFormatobject
Configuration for video output format.
| type* | const | "video" | |
| delivery | enum | The delivery mode for the video output. | inline | uri |
| aspect_ratio | enum | The aspect ratio for the video output. | 16:9 | 9:16 |
| duration | enum | The duration for the video output, in whole seconds. gemini-omni-1.2-flash generates 3s to 15s (https://aistudio.google.com/docs/models/gemini-omni-flash.md, Output video). | 3s | 4s | 5s | 6s | 7s | 8s · +7 more |
| resolution | enum | The video output resolution. Defaults to 720p. | 360p | 720p | 1080p | 4k |
WebhookConfigobject
Message for configuring webhook events for a request.
| uris | string[] | Optional. If set, these webhook URIs will be used for webhook events instead of the registered webhooks. | |
| user_metadata | object | Optional. The user metadata that will be returned on each event emission to the webhooks. |
WordInfoobject
Word-level ASR annotation for transcription output. Carries the word text, optional timing, and optional speaker attribution.
| type* | const | "word_info" | |
| text | string | The transcribed word. | |
| start_offset | string | Start offset in time of the word relative to the start of the audio. Present when timestamp_granularities contains "word". | format: google-duration |
| end_offset | string | End offset in time of the word relative to the start of the audio. Present when timestamp_granularities contains "word". | format: google-duration |
| speaker | string | Optional. Speaker label for this word (e.g. "spk_1", "spk_2"). Present when diarization_mode is set in TranscriptionConfig. | |
| start_index | integer | Start of segment of the response that is attributed to this source. | format: int32 |
| end_index | integer | End of the attributed segment, exclusive. | format: int32 |
Validate a payload
before you spend tokens$ curl -X POST https://modelschemas.com/v1/validate \
-d '{"provider":"gemini","endpointId":"gemini-omni-1.2-flash","payload":{…}}'
→ {"valid": false, "errors": [{"path": "#", "keyword": "required", …}]}