encoding/asn1: unmarshal of sequence of oid into slice of RawValue

[thsnr]

What version of Go are you using (go version)?

go version go1.7.1 linux/amd64

What operating system and processor architecture are you using (go env)?

GOOS="linux"
GOARCH="amd64"

What did you do?

Trying to ASN.1 unmarshal a sequence of object identifiers into a slice of asn1.RawValue:
https://play.golang.org/p/gWzGulJAow

What did you expect to see?

Expected the variable bad to contain a single asn1.RawValue element with the raw encoding of the object identifier.

What did you see instead?

asn1: structure error: sequence tag mismatch

[jcmturner]

Hi, Is there any update on this issue as I'm also having a problem where I need to unmarshal into an []asn1.RawValue

Thanks

[m4ns0ur]

@thsnr / @jcmturner

func main() {
	var good asn1.RawValue
	asn1.Unmarshal([]byte(der), &good)
	fmt.Println(good)
}

[thsnr]

var good asn1.RawValue

This results in good containing the entire ASN.1 SEQUENCE. We wish to unmarshal the SEQUENCE, just not individual elements in it.

[m4ns0ur]

@thsnr the struct has un-marshaled SEQUENCE (tag/value):

func main() {
	var good asn1.RawValue
	asn1.Unmarshal([]byte(der), &good)

        fmt.Println(good.Tag)
        fmt.Println(good.Bytes)
}

// 16
// [6 1 42]

so what are you expecting to get?

[thsnr]

I expect to get a slice of asn1.RawValues with a single element that has Tag 6 and Bytes 42.

Perhaps a Playground with a sequence of two elements gives a better example:
https://play.golang.org/p/QcF5Fw9LvG

Here I would expect to get a slice with two elements, the asn1.RawValue for OBJECT IDENTIFIER {1 2} and the asn1.RawValue for OBJECT IDENTIFIER {3 4}.

[m4ns0ur]

@thsnr I see your point now.

From godoc:

An ASN.1 SEQUENCE OF x or SET OF x can be written to a slice if an x can be written to the slice's element type.

So you can easily have a slice of OBJECT IDENTIFIER ([]asn1.ObjectIdentifier) like this:
https://play.golang.org/p/77B7jLWeRb

But about asn1.RawValue it's kind of confusing for Unmarshal, while it points to SEQUENCE itself:
https://play.golang.org/p/Zngp7rc4o3

Using []asn1.RawValue means, having SEQUENCE of SEQUENCEs (I changed your DER):
https://play.golang.org/p/WLf8eQIywc

IMO it's a correct behavior. I leave it to @bradfitz.

[thsnr]

tl;dr: We actually worked around this even before the bug was reported, so I am fine with leaving the current behavior, but it seems to go against the documentation.

I will add a little background. While in any sane situation I would just use []asn1.ObjectIdenfitier, we were implementing Cryptographic Message Syntax (CMS). One of the types there is Attribute (https://tools.ietf.org/html/rfc5652#page-14):

Attribute ::= SEQUENCE {
attrType OBJECT IDENTIFIER,
attrValues SET OF AttributeValue }

AttributeValue ::= ANY

So we have to unmarshal a SET OF ANY and one of the possible AttributeValues, Content Type (https://tools.ietf.org/html/rfc5652#section-11.1), was an OBJECT IDENTIFIER, which failed.

We worked around this back last year before this issue was reported (also by just unmarshaling into a single asn1.RawValue and going from there), but I still thought to report it as it seems to go against the same godoc snippet that @m4ns0ur quoted:

An ASN.1 SEQUENCE OF x or SET OF x can be written to a slice if an x can be written to the slice's element type.

Since we can unmarshal an OBJECT IDENTIFIER into an asn1.RawValue, this sentence implies that we can unmarshal SEQUENCE OF OBJECT IDENTIFIER into []asn1.RawValue.

Also as @m4ns0ur noted, unmarshaling a SEQUENCE OF SEQUENCE into []asn1.RawValue seems to work: we use it for unmarshaling a SET OF X.509 Certificates so that their FullBytes can be passed to x509.ParseCertificate. So this documentation sentence with regards to asn1.RawValue sometimes holds and sometimes not. We have not tested any other non-SEQUENCE types.

As I said, we have already worked around this situation, so it fine by me to leave the behavior as-is. :)

[m4ns0ur]

@thsnr sounds reasonable, let me look more.

[gopherbot]

Change https://golang.org/cl/84095 mentions this issue: testing: add testing on Unmarshal slice of RawValue

[m4ns0ur]

@thsnr so it's already fixed at a82ee9c, and you're gonna have it in Go1.10.
Just added a test case too. Thanks for reporting.