This extension can be used to unmarshall a Cloud Logging LogEntry message type.
| Status | |
|---|---|
| Stability | alpha |
| Distributions | contrib |
| Issues |   |
| Code Owners | @constanca-m |
| Emeritus | @alexvanboxel |
The extension expects each log to take up 1 line, and it will decode as many logs as log lines received.
Currently, this extension can parse the following logs into log record attributes:
- Cloud audit logs (extension mapping)
- VPC flow logs (extension mapping)
- Application Load Balancer logs(extension mapping)
- Global External Application Load Balancer
- Regional External Application Load Balancer
- Cloud Armor logs (embedded within load balancer logs) (extension mapping)
- Proxy Network Load Balancer logs (extension mapping)
- Cloud DNS logs (extension mapping)
- Passthrough Network Load Balancer logs (extension mapping)
For all others logs, the payload will be placed in the log record attribute. In this case, the following configuration options are supported:
handle_json_payload_as(Optional): This controls how the json payload of the log entry is parsed into the body. The defaultjsonparses it as standard JSON, whiletextwill the put the payload as a single string.handle_proto_payload_as(Optional): This controls how the json payload of the log entry is parsed into the body. The defaultjsonparses it as standard JSON, whileprotowill use the well known protobuf types used in a log entry providing a better type handling, but sacrificing stability. Usingtextwill the put the payload as a single string.
The log entry fields are mapped this way in the encoding:
| Original JSON Field Name | OpenTelemetry log field |
|---|---|
receiveTimestamp |
Log record field: observedTimeUnixNano |
timestamp |
Log record field: timeUnixNano |
insertId |
Log record attribute: log.record.uid |
logName |
Parse it and place it in the resource log attributes, if present: 1. gcp.project2. gcp.organization3. gcp.billing_account4. gcp.folder5. cloud.resource_id |
severity |
Parse it and place in log record fields: 1. severityNumber2. severityText |
trace |
Log record field: traceId |
spanId |
Log record field: spanId |
traceSampled |
Log record field: flags |
labels |
Log record attribute: gcp.label.<label_key> |
httpRequest.requestMethod |
Log record attribute: http.request.method |
httpRequest.requestUrl |
Log record attribute: url.fullParse it, and place it in the log record attributes: 1. url.path2. url.query3. url.domain |
httpRequest.requestSize |
Log record attribute: http.request.size |
httpRequest.status |
Log record attribute: http.response.status_code |
httpRequest.responseSize |
Log record attribute: http.response.size |
httpRequest.userAgent |
Log record attribute: user_agent.original |
httpRequest.remoteIp |
Log record attribute: network.peer.address |
httpRequest.serverIp |
Log record attribute: server.address |
httpRequest.referer |
Log record attribute: http.request.header.referer |
httpRequest.latency |
Log record attribute: http.request.server.duration |
httpRequest.cacheLookup |
Log record attribute: gcp.cache.lookup |
httpRequest.cacheHit |
Log record attribute: gcp.cache.hit |
httpRequest.cacheValidatedWithOriginServer |
Log record attribute: gcp.cache.validated_with_origin_server |
httpRequest.cacheFillBytes |
Log record attribute: gcp.cache.fill_bytes |
httpRequest.protocol |
Parse it, and place it in log record attribute: 1. network.protocol.name.2. network.protocol.version |
resource.type |
Resource log attribute: gcp.resource_type |
resource.labels |
Resource log attribute: gcp.label.<label_key> |
operation.id |
Log record attribute: gcp.operation.id |
operation.producer |
Log record attribute: gcp.operation.producer |
operation.first |
Log record attribute: gcp.operation.first |
operation.last |
Log record attribute: gcp.operation.last |
sourceLocation.file |
Log record attribute: code.file.path |
sourceLocation.line |
Log record attribute: code.line.number |
sourceLocation.function |
Log record attribute: code.function.name |
protoPayload |
Placed on the record body as is, unless log type is supported |
textPayload |
Placed on the record body as is, unless log type is supported |
jsonPayload |
Placed on the record body as is, unless log type is supported |
split.uid |
Log record attribute: gcp.split.uid |
split.index |
Log record attribute: gcp.split.index |
split.totalSplits |
Log record attribute: gcp.split.total |
errorGroups[].id |
Log record attribute: gcp.error_groups listEach element has attribute id |
apphub.application.container |
Log record attribute: gcp.apphub.application.container |
apphub.application.location |
Log record attribute: gcp.apphub.application.location |
apphub.application.id |
Log record attribute: gcp.apphub.application.id |
apphub.service.id |
Log record attribute: gcp.apphub.service.id |
apphub.service.environmentType |
Log record attribute: gcp.apphub.service.environment_type |
apphub.service.criticalityType |
Log record attribute: gcp.apphub.service.criticality_type |
apphub.workload.id |
Log record attribute: gcp.apphub.workload.id |
apphub.workload.environmentType |
Log record attribute: gcp.apphub.workload.environment_type |
apphub.workload.criticalityType |
Log record attribute: gcp.apphub.workload.criticality_type |
apphubDestination.application.container |
Log record attribute: gcp.apphub_destination.application.container |
apphubDestination.application.location |
Log record attribute: gcp.apphub_destination.application.location |
apphubDestination.application.id |
Log record attribute: gcp.apphub_destination.application.id |
apphubDestination.service.id |
Log record attribute: gcp.apphub_destination.service.id |
apphubDestination.service.environmentType |
Log record attribute: gcp.apphub_destination.service.environment_type |
apphubDestination.service.criticalityType |
Log record attribute: gcp.apphub_destination.service.criticality_type |
apphubDestination.workload.id |
Log record attribute: gcp.apphub_destination.workload.id |
apphubDestination.workload.environmentType |
Log record attribute: gcp.apphub_destination.workload.environment_type |
apphubDestination.workload.criticalityType |
Log record attribute: gcp.apphub_destination.workload.criticality_type |
The severity is mapped from Google Cloud Log Severity like this:
| CloudLog | Severity Number | CloudLog Description |
|---|---|---|
DEFAULT(0) |
UNSPECIFIED(0) |
The log entry has no assigned severity level. |
DEBUG(100) |
DEBUG(5) |
Debug or trace information. |
INFO(200) |
INFO(9) |
Routine information, such as ongoing status or performance. |
NOTICE(300) |
INFO2(10) |
Normal but significant events, such as start up, shut down, or a configuration change. |
WARNING(400) |
WARN(13) |
Warning events might cause problems. |
ERROR(500) |
ERROR(17) |
Error events are likely to cause problems. |
CRITICAL(600) |
FATAL(21) |
Critical events cause more severe problems or outages. |
ALERT(700) |
FATAL2(22) |
A person must take an action immediately. |
EMERGENCY(800) |
FATAL4(24) |
One or more systems are unusable. |
Currently, these are the log types that are specifically parsed into log record attributes.
A subset of logs processed by this extension are automatically tagged with an encoding.format attribute at the scope level to identify the source format. This allows you to easily filter and route logs based on their Google Cloud service origin.
The pattern used is gcp.<format_name>.
Examples:
- Audit Logs:
encoding.format: "gcp.auditlog" - VPC Flow Logs:
encoding.format: "gcp.vpcflow" - Application Load Balancer Logs:
encoding.format: "gcp.load-balacer" - Proxy Network Load Balancer Logs:
encoding.format: "gcp.proxy-nlb" - Cloud DNS Logs:
encoding.format: "gcp.dns" - Passthrough Network Load Balancer Logs:
encoding.format: "gcp.passthrough-nlb"
The encoding.format attribute is automatically determined based on the log type extracted from the logName field. The extension uses the following logic:
- Parse the logName: The extension extracts the log type from the
logNamefield.
For example, projects/my-project/logs/cloudaudit.googleapis.com%2Fsystem_event is identified as a system event log via the log type suffix cloudaudit.googleapis.com%2Fsystem_event.
-
Map log type to format: The extension maps specific log types to their corresponding encoding formats (
encoding.format):- Audit logs (activity, data access, system event, policy):
gcp.auditlog - VPC flow logs (network management-sourced and compute-sourced VPC flow logs):
gcp.vpcflow - Application Load Balancer logs (Global External and Regional External):
gcp.load-balancer - Cloud Armor logs (embedded within load balancer logs):
gcp.armorlog - Proxy Network Load Balancer logs:
gcp.proxy-nlb - Cloud DNS logs:
gcp.dns
- Audit logs (activity, data access, system event, policy):
-
Set the attribute: For recognized log types, the
encoding.formatattribute is set as an attribute of thescopefield in the OTEL output log, allowing for flexible filtering and routing.
For unrecognized log types, no encoding.format attribute is set.
The following format values are supported in the googlecloudlogentryencodingextension to identify different Google Cloud log types:
| GCP Log Type | Format Value | Description |
|---|---|---|
| Audit Logs | auditlog |
Google Cloud audit logs (activity, data access, system event, policy) |
| VPC Flow Logs | vpcflow |
Virtual Private Cloud flow log records |
| Application Load Balancer Logs | load-balancer |
Global and Regional External Application Load Balancer logs |
| Armor Logs | armorlog |
Google Cloud armor logs (security policies applied) |
| Proxy Network Load Balancer Logs | proxy-nlb |
Proxy Network Load Balancer connection logs |
| Cloud DNS Logs | dns |
Cloud DNS query and response logs |
| Passthrough Network Load Balancer Logs | passthrough-nlb |
Passthrough Network Load Balancer flow logs |
See the struct of the Cloud Audit Log payload in AuditLog. The fields are mapped this way in the extension:
| Original field | Log record attribute |
|---|---|
serviceName |
gcp.audit.service.name |
methodName |
gcp.audit.method.name |
resourceName |
gcp.audit.resource.name |
resourceLocation.currentLocations |
gcp.audit.resource.location.current |
resourceLocation.originalLocations |
gcp.audit.resource.location.original |
resourceOriginalState |
Currently not supported |
numResponseItems |
gcp.audit.response.items |
status.code |
rpc.jsonrpc.error_code |
status.message |
rpc.jsonrpc.error_message |
status.details |
Currently not supported |
authenticationInfo.principalEmail |
user.email |
authenticationInfo.authoritySelector |
gcp.audit.authentication.authority_selector |
authenticationInfo.thirdPartyPrincipal |
Currently not supported |
authenticationInfo.serviceAccountKeyName |
gcp.audit.authentication.service_account.key.name |
authenticationInfo.serviceAccountDelegationInfo |
Currently not supported |
authenticationInfo.principalSubject |
user.id |
authorizationInfo[*].resource |
Item entry resource in map gcp.audit.authorization |
authorizationInfo[*].permission |
Item entry permission in map gcp.audit.authorization |
authorizationInfo[*].granted |
Item entry granted in map `gcp.audit.authorization`` |
authorizationInfo.resourceAttributes |
Currently not supported |
policyViolationInfo.orgPolicyViolationInfo.payload |
Currently not supported |
policyViolationInfo.orgPolicyViolationInfo.resourceType |
gcp.audit.policy_violation.resource.type |
policyViolationInfo.orgPolicyViolationInfo.resourceTags |
gcp.audit.policy_violation.resource.tags |
policyViolationInfo.orgPolicyViolationInfo.violationInfo[*].constraint |
Item entry constraint in map gcp.audit.policy_violation.info |
policyViolationInfo.orgPolicyViolationInfo.violationInfo[*].errorMessage |
Item entry error_message in map gcp.audit.policy_violation.info |
policyViolationInfo.orgPolicyViolationInfo.violationInfo[*].checkedValue |
Item entry checked_value in map gcp.audit.policy_violation.info |
policyViolationInfo.orgPolicyViolationInfo.violationInfo[*].policyType |
Item entry policy_type in map gcp.audit.policy_violation.info |
requestMetadata.callerIp |
client.address |
requestMetadata.callerSuppliedUserAgent |
user_agent.original |
requestMetadata.callerNetwork |
gcp.audit.request.caller.network |
requestMetadata.requestAttributes.id |
http.request.id |
requestMetadata.requestAttributes.method |
http.request.method |
requestMetadata.requestAttributes.headers |
http.request.header.<header name> |
requestMetadata.requestAttributes.path |
url.path |
requestMetadata.requestAttributes.host |
http.request.header.host |
requestMetadata.requestAttributes.scheme |
url.scheme |
requestMetadata.requestAttributes.query |
url.query |
requestMetadata.requestAttributes.time |
gcp.audit.request.time |
requestMetadata.requestAttributes.size |
http.request.size |
requestMetadata.requestAttributes.protocol |
network.protocol.name |
requestMetadata.requestAttributes.reason |
gcp.audit.request.reason |
requestMetadata.requestAttributes.auth.principal |
gcp.audit.request.auth.principal |
requestMetadata.requestAttributes.auth.audiences |
gcp.audit.request.auth.audiences |
requestMetadata.requestAttributes.auth.presenter |
gcp.audit.request.auth.presenter |
requestMetadata.requestAttributes.auth.accessLevels |
gcp.audit.request.auth.access_levels |
requestMetadata.requestAttributes.auth.claims |
Currently not supported |
requestMetadata.destinationAttributes.ip |
server.address |
requestMetadata.destinationAttributes.port |
server.port |
requestMetadata.destinationAttributes.labels |
gcp.audit.destination.label.<label_key> |
requestMetadata.destinationAttributes.principal |
gcp.audit.destination.principal |
requestMetadata.destinationAttributes.regionCode |
gcp.audit.destination.region_code |
request |
Currently not supported |
response |
Currently not supported |
metadata |
Currently not supported |
serviceData |
[GCP Deprecated field] Currently not supported |
VPC flow logs are mapped this way in the resulting OpenTelemetry log:
| Flow log field | Attribute in OpenTelemetry log | Support |
|---|---|---|
connection.protocol |
network.transport |
supported |
connection.src_ip |
source.address |
supported |
connection.dest_ip |
destination.address |
supported |
connection.src_port |
source.port |
supported |
connection.dest_port |
destination.port |
supported |
reporter |
gcp.vpc.flow.reporter |
supported |
rtt_msec |
gcp.vpc.flow.network.rtt_ms |
supported |
round_trip_time.median_msec |
gcp.vpc.flow.rtt.median |
not yet supported |
bytes_sent |
gcp.vpc.flow.bytes_sent |
supported |
packets_sent |
gcp.vpc.flow.packets_sent |
supported |
start_time |
gcp.vpc.flow.start_time |
supported |
end_time |
gcp.vpc.flow.end_time |
supported |
src_gateway.project_id |
gcp.vpc.flow.source.gateway.project.id |
not yet supported |
src_gateway.location |
gcp.vpc.flow.source.gateway.region |
not yet supported |
src_gateway.name |
gcp.vpc.flow.source.gateway.name |
not yet supported |
src_gateway.type |
gcp.vpc.flow.source.gateway.type |
not yet supported |
src_gateway.vpc.project_id |
gcp.vpc.flow.source.gateway.vpc.project.id |
not yet supported |
src_gateway.vpc.subnetwork_name |
gcp.vpc.flow.source.gateway.vpc.subnet.name |
not yet supported |
src_gateway.vpc.subnetwork_region |
gcp.vpc.flow.source.gateway.vpc.subnet.region |
not yet supported |
src_gateway.vpc.vpc_name |
gcp.vpc.flow.source.gateway.vpc.name |
not yet supported |
src_gateway.interconnect_name |
gcp.vpc.flow.source.gateway.interconnect.name |
not yet supported |
src_gateway.interconnect_project_number |
gcp.vpc.flow.source.gateway.interconnect.project.number |
not yet supported |
dest_gateway.project_id |
gcp.vpc.flow.destination.gateway.project.id |
not yet supported |
dest_gateway.location |
gcp.vpc.flow.destination.gateway.region |
not yet supported |
dest_gateway.name |
gcp.vpc.flow.destination.gateway.name |
not yet supported |
dest_gateway.type |
gcp.vpc.flow.destination.gateway.type |
not yet supported |
dest_gateway.vpc.project_id |
gcp.vpc.flow.destination.gateway.vpc.project.id |
not yet supported |
dest_gateway.vpc.subnetwork_name |
gcp.vpc.flow.destination.gateway.vpc.subnet.name |
not yet supported |
dest_gateway.vpc.subnetwork_region |
gcp.vpc.flow.destination.gateway.vpc.subnet.region |
not yet supported |
dest_gateway.vpc.vpc_name |
gcp.vpc.flow.destination.gateway.vpc.name |
not yet supported |
dest_gateway.interconnect_name |
gcp.vpc.flow.destination.gateway.interconnect.name |
not yet supported |
dest_gateway.interconnect_project_number |
gcp.vpc.flow.destination.gateway.interconnect.project.number |
not yet supported |
src_gke_details.cluster.cluster_location |
gcp.vpc.flow.source.gke.cluster.location |
not yet supported |
src_gke_details.cluster.cluster_name |
gcp.vpc.flow.source.gke.cluster.name |
not yet supported |
src_gke_details.pod.pod_name |
gcp.vpc.flow.source.gke.pod.name |
not yet supported |
src_gke_details.pod.pod_namespace |
gcp.vpc.flow.source.gke.pod.namespace |
not yet supported |
src_gke_details.pod.pod_workload.workload_name |
gcp.vpc.flow.source.gke.pod.workload.name |
not yet supported |
src_gke_details.pod.pod_workload.workload_type |
gcp.vpc.flow.source.gke.pod.workload.type |
not yet supported |
src_gke_details.service.service_name |
gcp.vpc.flow.source.gke.service.name |
not yet supported |
src_gke_details.service.service_namespace |
gcp.vpc.flow.source.gke.service.namespace |
not yet supported |
dest_gke_details.cluster.cluster_location |
gcp.vpc.flow.destination.gke.cluster.location |
not yet supported |
dest_gke_details.cluster.cluster_name |
gcp.vpc.flow.destination.gke.cluster.name |
not yet supported |
dest_gke_details.pod.pod_name |
gcp.vpc.flow.destination.gke.pod.name |
not yet supported |
dest_gke_details.pod.pod_namespace |
gcp.vpc.flow.destination.gke.pod.namespace |
not yet supported |
dest_gke_details.pod.pod_workload.workload_name |
gcp.vpc.flow.destination.gke.pod.workload.name |
not yet supported |
dest_gke_details.pod.pod_workload.workload_type |
gcp.vpc.flow.destination.gke.pod.workload.type |
not yet supported |
dest_gke_details.service.service_name |
gcp.vpc.flow.destination.gke.service.name |
not yet supported |
dest_gke_details.service.service_namespace |
gcp.vpc.flow.destination.gke.service.namespace |
not yet supported |
src_google_service.type |
gcp.vpc.flow.source.google_service.type |
supported |
src_google_service.service_name |
gcp.vpc.flow.source.google_service.name |
supported |
src_google_service.connectivity |
gcp.vpc.flow.source.google_service.connectivity |
supported |
src_google_service.private_domain |
gcp.vpc.flow.source.google_service.domain.private |
not yet supported |
dest_google_service.type |
gcp.vpc.flow.destination.google_service.type |
supported |
dest_google_service.service_name |
gcp.vpc.flow.destination.google_service.name |
supported |
dest_google_service.connectivity |
gcp.vpc.flow.destination.google_service.connectivity |
supported |
src_instance.project_id |
gcp.vpc.flow.source.instance.project.id |
supported |
src_instance.region |
gcp.vpc.flow.source.instance.vm.region |
supported |
src_instance.vm_name |
gcp.vpc.flow.source.instance.vm.name |
supported |
src_instance.zone |
gcp.vpc.flow.source.instance.vm.zone |
supported |
src_instance.managed_instance_group.name |
gcp.vpc.flow.source.instance.managed_instance_group.name |
supported |
src_instance.managed_instance_group.region |
gcp.vpc.flow.source.instance.managed_instance_group.region |
supported |
src_instance.managed_instance_group.zone |
gcp.vpc.flow.source.instance.managed_instance_group.zone |
supported |
dest_instance.project_id |
gcp.vpc.flow.destination.instance.project.id |
supported |
dest_instance.region |
gcp.vpc.flow.destination.instance.vm.region |
supported |
dest_instance.vm_name |
gcp.vpc.flow.destination.instance.vm.name |
supported |
dest_instance.zone |
gcp.vpc.flow.destination.instance.vm.zone |
supported |
dest_instance.managed_instance_group.name |
gcp.vpc.flow.destination.instance.managed_instance_group.name |
supported |
dest_instance.managed_instance_group.region |
gcp.vpc.flow.destination.instance.managed_instance_group.region |
supported |
dest_instance.managed_instance_group.zone |
gcp.vpc.flow.destination.instance.managed_instance_group.zone |
supported |
src_location.asn |
gcp.vpc.flow.source.asn |
supported |
src_location.city |
gcp.vpc.flow.source.geo.city |
supported |
src_location.continent |
gcp.vpc.flow.source.geo.continent |
supported |
src_location.country |
gcp.vpc.flow.source.geo.country.iso_code.alpha3 |
supported |
src_location.region |
gcp.vpc.flow.source.geo.region |
supported |
dest_location.asn |
gcp.vpc.flow.destination.asn |
supported |
dest_location.city |
gcp.vpc.flow.destination.geo.city |
supported |
dest_location.continent |
gcp.vpc.flow.destination.geo.continent |
supported |
dest_location.country |
gcp.vpc.flow.destination.geo.country.iso_code.alpha3 |
supported |
dest_location.region |
gcp.vpc.flow.destination.geo.region |
supported |
src_vpc.project_id |
gcp.vpc.flow.source.project.id |
supported |
src_vpc.subnetwork_name |
gcp.vpc.flow.source.subnet.name |
supported |
src_vpc.subnetwork_region |
gcp.vpc.flow.source.subnet.region |
supported |
src_vpc.vpc_name |
gcp.vpc.flow.source.vpc.name |
supported |
dest_vpc.project_id |
gcp.vpc.flow.destination.project.id |
supported |
dest_vpc.subnetwork_name |
gcp.vpc.flow.destination.subnet.name |
supported |
dest_vpc.subnetwork_region |
gcp.vpc.flow.destination.subnet.region |
supported |
dest_vpc.vpc_name |
gcp.vpc.flow.destination.vpc.name |
supported |
internet_routing_details.egress_as_path.as_details.asn |
gcp.vpc.flow.egress.as_paths Each element has a nested as_details array containing asn attributes |
supported |
load_balancing.forwarding_rule_project_id |
gcp.vpc.flow.load_balancing.forwarding_rule.project.id |
not yet supported |
load_balancing.reporter |
gcp.vpc.flow.load_balancing.reporter |
not yet supported |
load_balancing.type |
gcp.vpc.flow.load_balancing.type |
not yet supported |
load_balancing.scheme |
gcp.vpc.flow.load_balancing.scheme |
not yet supported |
load_balancing.url_map_name |
gcp.vpc.flow.load_balancing.url_map.name |
not yet supported |
load_balancing.forwarding_rule_name |
gcp.vpc.flow.load_balancing.forwarding_rule.name |
not yet supported |
load_balancing.backend_service_name |
gcp.vpc.flow.load_balancing.backend_service.name |
not yet supported |
load_balancing.backend_group_name |
gcp.vpc.flow.load_balancing.backend_group.name |
not yet supported |
load_balancing.backend_group_type |
gcp.vpc.flow.load_balancing.backend_group.type |
not yet supported |
load_balancing.backend_group_location |
gcp.vpc.flow.load_balancing.backend_group.location |
not yet supported |
load_balancing.vpc.project_id |
gcp.vpc.flow.load_balancing.vpc.project.id |
not yet supported |
load_balancing.vpc.subnetwork_name |
gcp.vpc.flow.load_balancing.vpc.subnet.name |
not yet supported |
load_balancing.vpc.subnetwork_region |
gcp.vpc.flow.load_balancing.vpc.subnet.region |
not yet supported |
load_balancing.vpc.vpc_name |
gcp.vpc.flow.load_balancing.vpc.name |
not yet supported |
network_service.dscp |
gcp.vpc.flow.network_service.dscp |
supported |
psc.reporter |
gcp.vpc.flow.private_service_connect.reporter |
not yet supported |
psc.psc_endpoint.project_id |
gcp.vpc.flow.private_service_connect.endpoint.project.id |
not yet supported |
psc.psc_endpoint.region |
gcp.vpc.flow.private_service_connect.endpoint.region |
not yet supported |
psc.psc_endpoint.psc_connection_id |
gcp.vpc.flow.private_service_connect.endpoint.private_service_connect.connection.id |
not yet supported |
psc.psc_endpoint.target_service_type |
gcp.vpc.flow.private_service_connect.endpoint.target_service_type |
not yet supported |
psc.psc_endpoint.vpc.project_id |
gcp.vpc.flow.private_service_connect.endpoint.vpc.project.id |
not yet supported |
psc.psc_endpoint.vpc.subnetwork_name |
gcp.vpc.flow.private_service_connect.endpoint.vpc.subnet.name |
not yet supported |
psc.psc_endpoint.vpc.subnetwork_region |
gcp.vpc.flow.private_service_connect.endpoint.vpc.subnet.region |
not yet supported |
psc.psc_endpoint.vpc.vpc_name |
gcp.vpc.flow.private_service_connect.endpoint.vpc.name |
not yet supported |
psc.psc_attachment.project_id |
gcp.vpc.flow.private_service_connect.attachment.project.id |
not yet supported |
psc.psc_attachment.region |
gcp.vpc.flow.private_service_connect.attachment.region |
not yet supported |
psc.psc_attachment.vpc.project_id |
gcp.vpc.flow.private_service_connect.attachment.vpc.project.id |
not yet supported |
psc.psc_attachment.vpc.subnetwork_name |
gcp.vpc.flow.private_service_connect.attachment.vpc.subnet.name |
not yet supported |
psc.psc_attachment.vpc.subnetwork_region |
gcp.vpc.flow.private_service_connect.attachment.vpc.subnet.region |
not yet supported |
psc.psc_attachment.vpc.vpc_name |
gcp.vpc.flow.private_service_connect.attachment.vpc.name |
not yet supported |
rdma_traffic_type |
gcp.vpc.flow.remote_direct_memory_access.traffic_type |
not yet supported |
Application Load Balancer logs (both Global External and Regional External) are identified by the presence of load balancer-specific fields in the jsonPayload. are mapped this way in the resulting OpenTelemetry log:
| Original field | Log record attribute |
|---|---|
statusDetails |
gcp.load_balancing.status_details |
loadBalancingScheme |
gcp.load_balancing.scheme |
backendTargetProjectNumber |
gcp.load_balancing.backend_target_project_number |
remoteIp |
network.peer.address |
proxyStatus |
gcp.load_balancing.proxy_status |
overrideResponseCode |
gcp.load_balancing.override_response_code |
errorService |
gcp.load_balancing.error_service |
cacheId |
gcp.load_balancing.cache.id |
cacheDecision |
gcp.load_balancing.cache.decision |
backendNetworkName |
gcp.load_balancing.backend_network_name |
authPolicyInfo.result |
gcp.load_balancing.auth_policy_info.result |
authPolicyInfo.policies |
gcp.load_balancing.auth_policy_info.policies |
authPolicyInfo.policies.name |
gcp.load_balancing.auth_policy_info.policies.name |
authPolicyInfo.policies.result |
gcp.load_balancing.auth_policy_info.policies.result |
authPolicyInfo.policies.details |
gcp.load_balancing.auth_policy_info.policies.details |
tls.earlyDataRequest |
tls.early_data_request |
tls.protocol |
tls.protocol.name |
tls.cipher |
tls.cipher |
mtls.clientCertSha256Fingerprint |
tls.client.hash.sha256 |
mtls.clientCertValidStartTime |
tls.client.not_before |
mtls.clientCertValidEndTime |
tls.client.not_after |
mtls.clientCertIssuerDn |
tls.client.issuer |
mtls.clientCertSubjectDn |
tls.client.subject |
mtls.clientCertChain |
tls.client.certificate_chain |
mtls.clientCertPresent |
mtls.client_cert.present |
mtls.clientCertChainVerified |
mtls.client_cert.chain_verified |
mtls.clientCertError |
mtls.client_cert.error |
mtls.clientCertSerialNumber |
mtls.client_cert.serial_number |
mtls.clientCertSpiffeId |
mtls.client_cert.spiffe_id |
mtls.clientCertURISans |
mtls.client_cert.uri_sans |
mtls.clientCertDnsnameSans |
mtls.client_cert.dnsname_sans |
mtls.clientCertLeaf |
mtls.client_cert.leaf |
Note: Application Load Balancer logs may contain embedded Cloud Armor security policy information. When present, these policies are parsed according to the Cloud Armor logs mapping (see below).
Cloud Armor logs are embedded within Application Load Balancer logs and are mapped this way in the resulting OpenTelemetry log.
| Original field | Log record attribute |
|---|---|
securityPolicyRequestData.recaptchaActionToken.score |
gcp.armor.request_data.recaptcha_action_token.score |
securityPolicyRequestData.recaptchaSessionToken.score |
gcp.armor.request_data.recaptcha_session_token.score |
securityPolicyRequestData.userIpInfo.source |
gcp.armor.request_data.user_ip.source |
securityPolicyRequestData.userIpInfo.ipAddress |
client.address |
securityPolicyRequestData.remoteIpInfo.ipAddress |
network.peer.address |
securityPolicyRequestData.remoteIpInfo.regionCode |
geo.region.iso_code |
securityPolicyRequestData.remoteIpInfo.asn |
gcp.armor.request_data.remote_ip.asn |
securityPolicyRequestData.tlsJa4Fingerprint |
tls.client.ja4 |
securityPolicyRequestData.tlsJa3Fingerprint |
tls.client.ja3 |
name |
gcp.armor.security_policy.name |
priority |
gcp.armor.security_policy.priority |
configuredAction |
gcp.armor.security_policy.configured_action |
outcome |
gcp.armor.security_policy.outcome |
rateLimitAction.key |
gcp.armor.security_policy.rate_limit.action.key |
rateLimitAction.outcome |
gcp.armor.security_policy.rate_limit.action.outcome |
adaptiveProtection.autoDeployAlertId |
gcp.armor.security_policy.adaptive_protection.auto_deploy.alert_id |
preconfiguredExprIds |
gcp.armor.security_policy.preconfigured.expr_ids |
threatIntelligence.categories |
gcp.armor.security_policy.threat_intelligence.categories |
addressGroup.names |
gcp.armor.security_policy.address_group.names |
Note: There are 4 different policy types (enforcedSecurityPolicy, previewSecurityPolicy, enforcedEdgeSecurityPolicy, previewEdgeSecurityPolicy). Each policy type creates a separate nested attribute structure (e.g., gcp.armor.security_policy.type.enforced, gcp.armor.security_policy.type.preview, gcp.armor.security_policy.type.enforced_edge, gcp.armor.security_policy.type.preview_edge) containing the security policy fields listed above. Multiple policy types can be present simultaneously in a single log entry, each represented by its own nested map. All fields explanations are available at Cloud Armor logs.
Proxy Network Load Balancer connection logs are mapped into OpenTelemetry attributes as follows:
| Original field | Log record attribute |
|---|---|
connection.clientIp |
client.address |
connection.clientPort |
client.port |
connection.serverIp |
server.address |
connection.serverPort |
server.port |
connection.protocol |
network.transport (translated from IANA protocol number, e.g., tcp, udp, icmp) |
startTime |
gcp.load_balancing.proxy_nlb.connection.start_time |
endTime |
gcp.load_balancing.proxy_nlb.connection.end_time |
serverBytesReceived |
gcp.load_balancing.proxy_nlb.server.bytes_received |
serverBytesSent |
gcp.load_balancing.proxy_nlb.server.bytes_sent |
Cloud DNS logs are mapped into OpenTelemetry attributes as follows:
| Original field | Log record attribute |
|---|---|
queryName |
dns.question.name |
queryType |
dns.question.type |
responseCode |
dns.response_code |
alias_query_response_code |
gcp.dns.alias_query.response.code |
authAnswer |
gcp.dns.auth_answer |
rdata |
dns.answer.data |
destinationIP |
server.address |
sourceNetwork |
gcp.dns.client.vpc.name |
source_type |
gcp.dns.client.type |
sourceIP |
client.address |
protocol |
network.transport |
location |
cloud.region |
target_name |
gcp.dns.server.name |
target_type |
gcp.dns.server.type |
serverLatency |
gcp.dns.server.latency |
egressError |
gcp.dns.egress.error |
healthyIps |
gcp.dns.healthy.ips |
unhealthyIps |
gcp.dns.unhealthy.ips |
dns64Translated |
gcp.dns.dns64.translated |
vmInstanceId |
host.id |
vmInstanceName |
host.name |
vmProjectId |
gcp.project.id |
vmZoneName |
cloud.availability_zone |
Protocol translation: The numeric protocol field from GCP is automatically translated to human-readable protocol names using the IANA Protocol Numbers standard. Common values include:
6→tcp17→udp1→icmp
Resource labels such as backend_name, network_name, and load_balancing_scheme are surfaced automatically via the existing gcp.label.* attribute pattern.
Passthrough Network Load Balancer flow logs cover both External and Internal Network Load Balancers. These logs are mapped into OpenTelemetry attributes as follows:
| Original field | Log record attribute |
|---|---|
connection.clientIp |
client.address |
connection.clientPort |
client.port |
connection.serverIp |
server.address |
connection.serverPort |
server.port |
connection.protocol |
network.transport (translated from IANA protocol number, e.g., tcp, udp, icmp) |
startTime |
gcp.load_balancing.passthrough_nlb.packets.start_time |
endTime |
gcp.load_balancing.passthrough_nlb.packets.end_time |
bytesReceived |
gcp.load_balancing.passthrough_nlb.bytes_received |
bytesSent |
gcp.load_balancing.passthrough_nlb.bytes_sent |
packetsReceived |
gcp.load_balancing.passthrough_nlb.packets_received |
packetsSent |
gcp.load_balancing.passthrough_nlb.packets_sent |
rtt |
gcp.load_balancing.passthrough_nlb.rtt |
Supported log types: This parser handles both:
type.googleapis.com/google.cloud.loadbalancing.type.ExternalNetworkLoadBalancerLogEntrytype.googleapis.com/google.cloud.loadbalancing.type.InternalNetworkLoadBalancerLogEntry
Protocol translation: The numeric protocol field from GCP is automatically translated to human-readable protocol names using the IANA Protocol Numbers standard. Common values include:
6→tcp17→udp1→icmp
Resource labels such as backend_group_name, backend_network_name, forwarding_rule_name, and region are set with the gcp.label.* prefix.