boringssl/crypto/x509/test/pss_sha256_salt_overflow.pem

Tighten up supported PSS combinations in X.509. Matching Chromium, Go, and TLS 1.3, only allow SHA-256, SHA-384, and SHA-512 RSA-PSS signatures, where MGF-1 and message hash match and salt length is hash length. Sadly, we are stuck tolerating an explicit trailerField for now. See the certificates in cl/362617931. This also fixes an overflow bug in handling the salt length. On platforms with 64-bit long and 32-bit int, we would misinterpret, e.g, 2^62 + 32 as 32. Also clean up the error-handling of maskHash. It was previously handled in a very confusing way; syntax errors in maskHash would succeed and only be noticed later, in rsa_mgf1_decode. I haven't done it in this change, but as a followup, we can, like Chromium, reduce X.509 signature algorithms down to a single enum. Update-Note: Unusual RSA-PSS combinations in X.509 are no longer accepted. This same change (actually a slightly stricter version) has already landed in Chrome. Bug: 489 Change-Id: I85ca3a4e14f76358cac13e66163887f6dade1ace Reviewed-on: https://boringssl-review.googlesource.com/c/boringssl/+/53865 Auto-Submit: David Benjamin <davidben@google.com> Reviewed-by: Adam Langley <agl@google.com> Commit-Queue: Adam Langley <agl@google.com> 3 years ago			`-----BEGIN CERTIFICATE-----`
			`MIIDGzCCAcigAwIBAgIBADBIBgkqhkiG9w0BAQowO6APMA0GCWCGSAFlAwQCAQUA`
			`oRwwGgYJKoZIhvcNAQEIMA0GCWCGSAFlAwQCAQUAogoCCEAAAAAAAAAgMBQxEjAQ`
			`BgNVBAMMCUJvcmluZ1NTTDAiGA8wMDAwMDEwMTAwMDAwMFoYDzk5OTkxMjMxMjM1`
			`OTU5WjAUMRIwEAYDVQQDDAlCb3JpbmdTU0wwggEiMA0GCSqGSIb3DQEBAQUAA4IB`
			`DwAwggEKAoIBAQC6C9qEGRIBQXV8Lj29vVu+U+tyXzSSinWIumK5ijPhCm3DLnv4`
			`RayxkFwemtnkGRZ/o94ZnsXkBfU/IlsYdkuq8wK9WI/ql3gwWjH+KARIhIQcSLGi`
			`JcLN6kGuG2nlRBKMcPgPiEq2B0yBXFf4tG3CBbeae7+8G7uvOmv8NLyKj32neWpn`
			`UCTL5o2VwyPoxjLxT5gUR69v9XSVFj2irCZbsEedeKSb++LqyMhLfnRTzNv+ZHNh`
			`4izZHrktR25MvnT5QyBq32hx7AjZ2/xo70OmH7w10a2DwsVjJNMdxTEmgyvU9M6C`
			`eYRPX1Ykfg+sXCTtkTVAlBDUviIqY95CKy25AgMBAAEwSAYJKoZIhvcNAQEKMDug`
			`DzANBglghkgBZQMEAgEFAKEcMBoGCSqGSIb3DQEBCDANBglghkgBZQMEAgEFAKIK`
			`AghAAAAAAAAAIAOCAQEAn7ufGUhwZUfP13AEXFZGo7tpWYKbiDpVnHTtj/21/SWd`
			`TXP1LJWIJuo8pGs/ZsXI+XoXJMk01G1wVPPUny/D7T9WXLW191UFzIGO0bJZIfv2`
			`M9YbRkTsCiAYUuyFUlwwSLMqMXrZjRXlgulv3DjWrDHrAqfst847Ety24P1uYG7C`
			`m4JV8Sa0SIKRntd00YYmk6oUZNgEzUps7moLsOEox3U2s6wTipl/++9H5CI5mQTS`
			`fdGMRzEsuJRfXkgMccEfDw2wvNfmNzILGDsvxjCilkEisuMPxlRptSk5agYFujAW`
			`D3QjJGCgGlSXhN3JuH9S2/0N5gRQpN/98beTTXpxvg==`
			`-----END CERTIFICATE-----`