Proposing new SPARQL 1.1 compliance tests

[RobinTF]

This is a follow-up issue to my comment in #243 (comment).
Here's a list of compliance tests I came up with, for review. If they don't contain any mistakes I plan on eventually creating a PR for each individual one:

Relative IRI

Dataset

default

Query

BASE <http://example.com/xxx/abc>
SELECT (<> AS ?res0) (<def> AS ?res1) (<#ghi> AS ?res2) (<./jkl> AS ?res3) (<../mno> AS ?res4) (</pqr> AS ?res5) {}

Expected Result

?res0	?res1	?res2	?res3	?res4	?res5
<http://example.com/xxx/abc>	<http://example.com/xxx/def>	<http://example.com/xxx/abc#ghi>	<http://example.com/xxx/jkl>	<http://example.com/mno>	<http://example.com/pqr>

Language Tags case insensitivity

Dataset

default

Query

SELECT (""@en = ""@EN AS ?res0) (LANG(""@en) = LANG(""@EN) AS ?res1) (STRLANG("", "en") = STRLANG("", "EN") AS ?res2) {}

Expected Result

?res0	?res1	?res2
true	true	true

Nested Graphs 1

Dataset

default

graph1.ttl

@prefix : <http://example.com/> .
:s :p :o
:s1 :p :o1

graph2.ttl

@prefix : <http://example.com/> .
:s :p :o
:s2 :p :o2

Query

PREFIX : <http://example.com/>
SELECT ?s ?o WHERE {
  GRAPH ?g {
    GRAPH ?g {
      ?s :p ?o
    }
  }
}
ORDER BY ?s ?o

Expected Result

?s	?o
<http://example.com/s> <http://example.com/o>
<http://example.com/s> <http://example.com/o>
<http://example.com/s1> <http://example.com/o1>
<http://example.com/s2> <http://example.com/o2>

Nested Graphs 2

Dataset

default

graph1.ttl

@prefix : <http://example.com/> .
:s :p :o
:s1 :p :o1

graph2.ttl

@prefix : <http://example.com/> .
:s :p :o
:s2 :p :o2

Query

PREFIX : <http://example.com/>
SELECT ?s ?o WHERE {
  GRAPH ?g1 {
    GRAPH ?g0 {
      ?s :p ?o
    }
  }
}
ORDER BY ?s ?o

Expected Result

?s	?o
<http://example.com/s> <http://example.com/o>
<http://example.com/s> <http://example.com/o>
<http://example.com/s> <http://example.com/o>
<http://example.com/s> <http://example.com/o>
<http://example.com/s1> <http://example.com/o1>
<http://example.com/s1> <http://example.com/o1>
<http://example.com/s2> <http://example.com/o2>
<http://example.com/s2> <http://example.com/o2>

Nested Graphs 3

Dataset

default

graph1.ttl

@prefix : <http://example.com/> .
:s :p :o
:s1 :p :o1

graph2.ttl

@prefix : <http://example.com/> .
:s :p :o
:s2 :p :o2

Query

PREFIX : <http://example.com/>
SELECT ?s ?o WHERE {
  GRAPH ?g {
    GRAPH <graph1.ttl> {
      ?s :p ?o
    }
  }
}
ORDER BY ?s ?o

Expected Result

?s	?o
<http://example.com/s> http://example.com/o>
<http://example.com/s> http://example.com/o>
<http://example.com/s1> http://example.com/o1>
<http://example.com/s1> http://example.com/o1>

Nested Graphs 4

Dataset

default

graph1.ttl

@prefix : <http://example.com/> .
:s :p :o
:s1 :p :o1

graph2.ttl

@prefix : <http://example.com/> .
:s :p :o
:s2 :p :o2

Query

PREFIX : <http://example.com/>
SELECT ?s ?o WHERE {
  GRAPH <graph1.ttl> {
    GRAPH ?g {
      ?s :p ?o
    }
  }
}
ORDER BY ?s ?o

Expected Result

?s	?o
<http://example.com/s> <http://example.com/o>
<http://example.com/s> <http://example.com/o>
<http://example.com/s1> <http://example.com/o1>
<http://example.com/s2> <http://example.com/o2>

Graph with VALUES

Dataset

default

graph1.ttl

@prefix : <> .
:s :p :o

graph2.ttl

@prefix : <> .
:s :p :o

Query

SELECT ?x ?g WHERE {
  GRAPH ?g {
    VALUES ?x { 1 2 }
  }
}
ORDER BY ?x ?g

Expected Result

?x	?g
1	<graph1.ttl>
1	<graph2.ttl>
2	<graph1.ttl>
2	<graph2.ttl>

Graph with LIMIT/OFFSET with VALUES

Dataset

default

graph1.ttl

@prefix : <> .
:s :p :o

graph2.ttl

@prefix : <> .
:s :p :o

Query

PREFIX : <http://example.com/>
SELECT ?x WHERE {
  GRAPH ?g {
    SELECT ?x {
      VALUES ?x { 1 2 3 }
    }
    ORDER BY ?x
    OFFSET 1
    LIMIT 1
  }
}

Expected Result

?x
2
2

Graph with LIMIT/OFFSET

Dataset

default

graph1.ttl

@prefix : <http://example.com/> .
:s :p :o
:s1 :p :o1
:s1 :p :o2
:s1 :p :o3

graph2.ttl

@prefix : <http://example.com/> .
:s :p :o
:s2 :p :o1
:s2 :p :o2

graph3.ttl

@prefix : <http://example.com/> .
:s :p :o

Query

PREFIX : <http://example.com/>
SELECT ?s ?o WHERE {
  GRAPH ?g {
    SELECT ?s ?o {
      ?s :p ?o
    }
    ORDER BY ?s ?o
    OFFSET 1
    LIMIT 2
  }
}
ORDER BY ?s ?o

Expected Result

?s	?o
<http://example.com/s1> <http://example.com/o1>
<http://example.com/s1> <http://example.com/o2>
<http://example.com/s2> <http://example.com/o1>
<http://example.com/s2> <http://example.com/o2>

[Tpt]

Thank you! MRs would be amazing.

Some small nits (the tests all looks great to me):

• I think a lot of these tests actually cover SPARQL 1.0 features and can live in the sparql1.0 directories (feel free to ask if you are unsure)
• No need of an explicit ORDER BY to check results: by default the testsuite SPARQL results are compared in an order-insensitive way.
• I think "graph with values" is already covered by SPARQL 1.1: VALUES wrapped in a GRAPH without BGP #234 but I might be wrong.

[RobinTF]

@Tpt Thanks for your feedback. Some comments from my side:

• Regarding the question of SPARQL 1.1 or 1.0, are there any 1.0 tests that are invalid for 1.1 engines? And if yes how would I know which tests to consider for a 1.1 engine? My previous understanding was that for 1.1 compliance I only have to look at the 1.1 directory, but if I understand you correctly that's not the case.
• Regarding the ORDER BY, makes sense, didn't think of that, but for the limit/offset tests it's probably still required.
• SPARQL 1.1: VALUES wrapped in a GRAPH without BGP #234 is very similar, but my explicit goal was to test a scenario where the graph variable does explicitly not exist within the graph clause, neither implicitly SPARQL 1.1: VALUES wrapped in a GRAPH without BGP #234 uses UNDEF in a particular case which is admittedly very similar, but I think it's different enough to justify another test.

[Tpt]

Regarding the question of SPARQL 1.1 or 1.0, are there any 1.0 tests that are invalid for 1.1 engines? And if yes how would I know which tests to consider for a 1.1 engine? My previous understanding was that for 1.1 compliance I only have to look at the 1.1 directory, but if I understand you correctly that's not the case.

I am not aware of any test valid for 1.0 but not 1.1, opposite to some 1.0/1.1 tests that are not valid in 1.2 anymore (e.g. the ones that checks that sameTerm("foo", "foo"^^xsd:string) is false). But I might be wrong. If you only look at the 1.1 directory you will miss a lot of testcases.

Regarding the ORDER BY, makes sense, didn't think of that, but for the limit/offset tests it's probably still required.

Yes! Definitely!

#234 is very similar, but my explicit goal was...

Make sense!

[RobinTF]

@Tpt Thanks for clarifying that, wouldn't it make sense in that case to make the connection explicit somehow? If you were to implement a SPARQL 1.2 engine it would be really beneficial to know which one of the 1.1 and 1.0 tests are still valid. And if you're doing that for 1.2 then it would also make sense to do the same for 1.1, right?

[Tpt]

@Tpt Thanks for clarifying that, wouldn't it make sense in that case to make the connection explicit somehow?

Yes, definitely! Feel free to open an issue for that!

[afs]

(e.g. the ones that checks that sameTerm("foo", "foo"^^xsd:string) is false)

Sorry - where's that in the test suite?

That's an RDF 1.0/1.1 change (datatype for plain literals) along with the introduction of rdf:langString.
:dawg-datatype-3 for example. https://github.com/w3c/rdf-tests/blob/main/sparql/sparql10/expr-builtin/result-datatype-3.srx which seems to have been RDF 1.1 when first added to rfd-tests.

IIRC all the tests here have updated for RDF 1.1 (i.e. from manifests).

e.g. dae46e5