Yes, the aws_access_key_id and aws_security_token (if present) are considered safe to expose.
To be able to do any harm, and attacker would theoretically need to reverse-engineer the third component, the aws access key secret, based on the request parameters and signature, so that they could generate valid alternate signatures for alternate requests.
If that were possible, the attacker could then perform any action that the temporary credentials have the authority to do -- however -- this would involve reverse-engineering multiple rounds of HMAC-SHA-256, and is considered computationally infeasible.
Additionally, when using temporary credentials (which is where you'd see aws_security_token), the credentials are valid only for a short time, anyway -- so even if reverse-engineering were practical, it would have to be done in an impractically short period of time.
The security token itself -- presumably -- is a signed and encrypted message describing the permissions that accompany the temporary access key id and secret, facilitating decentralization of the authorization decisions based on the tokens, within AWS. Regardless of its actual content (which doesn't appear to be documented), it isn't independently useful without the accompanying temporary access key id and secret -- and the secret is not disclosed in the signed URL and can't feasibly be reverse-engineered, as discussed above. The temporary access key id and secret, conversely, are useless without the token.