Skip to content

Commit 1951cc5

Browse files
vsuharnikovvsuharnikov
authored
Mining stopped because of endorsement and other fixes (#4031)
* Can't append a microblock after a microblock with conflict endorsement * New test: first microblock is invalid, next is valid and accepted * New test: more than max endorsements * New test: reject a second conflict endorsement * OneNodeFinalizationTestSuite stabilization * max-endorsements renamed to max-valid-endorsers * Finalization height must be positive * Max valid endorsers limit durign mining * Additional tests * Fixed warning * New tests * Updated tests * Cleanup: Blockchain.isMiningAllowed contains checks commitments, removed isCommitted * New test: not enough balance for finalization because of conflicting vote * Cleanup: simpler finalization calculation, new tests * Fixed invalid total block signature when reached, lost and reached again finalization * Fix * New test * Additional BLS tests * New BLS private key generation * Additional BLS tests * Debug logs * Debug logs (2) * Cleanup * New keyblock triggers finalization calculations * Fixing tests * Fixed wrong finalization height in sending endorsement, fixed tests * Docker tests fixes * Updated tests * finalizedBlockAt: 404 instead of null * Cleanup * Better logs * New test and logs * Fixed compilation issue, added compilePR sbt task to check compilation warnings * Routes fixes * Rescheduling miner after microblock, fixed bug with wrong balance during appending * GeneratorBalances renamed to GeneratorSet * min-micro-block-age: setting applied only on local node * minimal-block-generation-offset based on previous block timestamp * Challenging by a generator not from generator set * Anyone can challenge a bock * ChallengingAfterFinalizationSuite: the finalization header is empty in a challenging block * Fixed not starting mining because of minimal-block-generation-offset * Mining error without a stack trace * Don't broadcast an endorsement if not enough generating balance * Fixed no ports Docker issue after restart * Do not accept endorsements from poor endorsers * - Fixed sorting of endorsements - Additional filter (not required, but nice to have) * Fixed limit of max valid endorsements * Remove sbt-docker, build via external process * TwoNodesFinalizationTestSuite stabilization * TwoNodesFinalizationTestSuite stabilization (2) * Stabilization of tests relying on minimal-block-generation-offset * Temporarily disabled Qase (preventing jobs) * Right finalization height in endorsements
1 parent e36304d commit 1951cc5

File tree

93 files changed

+2843
-1035
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

93 files changed

+2843
-1035
lines changed

‎.github/workflows/check-pr.yaml‎

Lines changed: 25 additions & 24 deletions
Original file line numberDiff line numberDiff line change
@@ -30,30 +30,31 @@ jobs:
3030
find node/tests/target/test-reports -name "*.xml" -exec sed -E -i \
3131
-e 's/time="(0(\.0+)?|-0\.[0-9]+)"/time="0.001"/g' \
3232
-e 's/name="([^"]*)(NODE-[0-9]+ )([^"]*)"/name="\2 \1\3"/g' {} +
33-
- uses: qase-tms/gh-actions/run-create@v1
34-
if: always()
35-
id: qase-run-create
36-
with:
37-
token: ${{ secrets.QASE_API_TOKEN }}
38-
project: NODE
39-
title: Check PR ${{ github.run_id }} — ${{ github.head_ref || github.ref_name }}
40-
description: |
41-
PR [#${{ github.event.number }}](https://github.com/wavesplatform/Waves/pulls/${{ github.event.number }}) — Run [#${{ github.run_number }}](https://github.com/wavesplatform/Waves/actions/runs/${{ github.run_id }})
42-
- uses: qase-tms/gh-actions/report@v1
43-
if: always() && steps.qase-run-create.outcome == 'success'
44-
with:
45-
token: ${{ secrets.QASE_API_TOKEN }}
46-
project: NODE
47-
id: ${{ steps.qase-run-create.outputs.id }}
48-
format: junit
49-
path: node/tests/target/test-reports
50-
batch: 100
51-
- uses: qase-tms/gh-actions/run-complete@v1
52-
if: always() && steps.qase-run-create.outcome == 'success'
53-
with:
54-
token: ${{ secrets.QASE_API_TOKEN }}
55-
project: NODE
56-
id: ${{ steps.qase-run-create.outputs.id }}
33+
# Temporarily disabled because of 402 Payment Required
34+
# - uses: qase-tms/gh-actions/run-create@v1
35+
# if: always()
36+
# id: qase-run-create
37+
# with:
38+
# token: ${{ secrets.QASE_API_TOKEN }}
39+
# project: NODE
40+
# title: Check PR ${{ github.run_id }} — ${{ github.head_ref || github.ref_name }}
41+
# description: |
42+
# PR [#${{ github.event.number }}](https://github.com/wavesplatform/Waves/pulls/${{ github.event.number }}) — Run [#${{ github.run_number }}](https://github.com/wavesplatform/Waves/actions/runs/${{ github.run_id }})
43+
# - uses: qase-tms/gh-actions/report@v1
44+
# if: always() && steps.qase-run-create.outcome == 'success'
45+
# with:
46+
# token: ${{ secrets.QASE_API_TOKEN }}
47+
# project: NODE
48+
# id: ${{ steps.qase-run-create.outputs.id }}
49+
# format: junit
50+
# path: node/tests/target/test-reports
51+
# batch: 100
52+
# - uses: qase-tms/gh-actions/run-complete@v1
53+
# if: always() && steps.qase-run-create.outcome == 'success'
54+
# with:
55+
# token: ${{ secrets.QASE_API_TOKEN }}
56+
# project: NODE
57+
# id: ${{ steps.qase-run-create.outputs.id }}
5758
- uses: scacap/action-surefire-report@5609ce4db72c09db044803b344a8968fd1f315da
5859
if: always()
5960
with:

‎benchmark/src/test/scala/com/wavesplatform/RollbackBenchmark.scala‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -84,7 +84,7 @@ object RollbackBenchmark extends ScorexLogging {
8484
computedBlockStateHash = ByteStr.empty,
8585
genesisBlock,
8686
newFinalizedHeight = GenesisBlockHeight,
87-
generatorBalances = Seq.empty
87+
generatorSet = Seq.empty
8888
)
8989

9090
val nextBlock =
@@ -117,7 +117,7 @@ object RollbackBenchmark extends ScorexLogging {
117117
computedBlockStateHash = ByteStr.empty,
118118
nextBlock,
119119
newFinalizedHeight = GenesisBlockHeight,
120-
generatorBalances = Seq.empty
120+
generatorSet = Seq.empty
121121
)
122122

123123
log.info("Rolling back")

‎benchmark/src/test/scala/com/wavesplatform/state/BaseState.scala‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -90,7 +90,7 @@ trait BaseState {
9090
differResult.computedStateHash,
9191
next,
9292
newFinalizedHeight = GenesisBlockHeight,
93-
generatorBalances = Seq.empty
93+
generatorSet = Seq.empty
9494
)
9595
}
9696

‎build.sbt‎

Lines changed: 33 additions & 13 deletions
Original file line numberDiff line numberDiff line change
@@ -247,10 +247,29 @@ buildRIDERunnerForDocker := {
247247
)
248248
}
249249

250-
lazy val checkPRRaw = taskKey[Unit]("Build a project and run unit tests")
251-
checkPRRaw := Def
250+
lazy val compilePRRaw = taskKey[Unit]("Compile the project")
251+
compilePRRaw := Def
252252
.sequential(
253253
clean,
254+
Def.task {
255+
(`lang-tests` / Test / compile).value
256+
(`repl-jvm` / Test / compile).value
257+
(`lang-tests-js` / Test / compile).value
258+
(`grpc-server` / Test / compile).value
259+
(`node-tests` / Test / compile).value
260+
(`node-it` / Test / compile).value
261+
(benchmark / Test / compile).value
262+
(`node-generator` / Compile / compile).value
263+
(`ride-runner` / Test / compile).value
264+
(`lang-jvm` / Test / compile).value
265+
}
266+
)
267+
.value
268+
269+
lazy val checkPRRaw = taskKey[Unit]("Compile the project and run unit tests")
270+
checkPRRaw := Def
271+
.sequential(
272+
compilePRRaw,
254273
Def.task {
255274
(`lang-tests` / Test / test).value
256275
(`repl-jvm` / Test / test).value
@@ -259,9 +278,6 @@ checkPRRaw := Def
259278
(`grpc-server` / Test / test).value
260279
(`node-tests` / Test / test).value
261280
(`repl-js` / Compile / fullOptJS).value
262-
(`node-it` / Test / compile).value
263-
(benchmark / Test / compile).value
264-
(`node-generator` / Compile / compile).value
265281
(`ride-runner` / Test / test).value
266282
(node / assembly).value
267283
buildTarballsForDocker.value
@@ -270,16 +286,20 @@ checkPRRaw := Def
270286
)
271287
.value
272288

273-
def checkPR: Command = Command.command("checkPR") { state =>
274-
val newState = Project
275-
.extract(state)
276-
.appendWithoutSession(
289+
def commandWithFatalWarnings(commandName: String, task: TaskKey[Unit]): Command =
290+
Command.command(commandName) { state =>
291+
val extracted = Project.extract(state)
292+
val newState = extracted.appendWithoutSession(
277293
Seq(Global / scalacOptions ++= Seq("-Xfatal-warnings")),
278294
state
279295
)
280-
Project.extract(newState).runTask(checkPRRaw, newState)
281-
state
282-
}
296+
297+
Project.extract(newState).runTask(task, newState)
298+
state
299+
}
300+
301+
def compilePR = commandWithFatalWarnings("compilePR", compilePRRaw)
302+
def checkPR = commandWithFatalWarnings("checkPR", checkPRRaw)
283303

284304
lazy val completeQaseRun = taskKey[Unit]("Complete Qase run")
285305
completeQaseRun := Def.task {
@@ -347,4 +367,4 @@ def generateGenesisCommand: Command =
347367
state
348368
}
349369

350-
commands ++= Seq(checkPR, buildReleaseArtifacts, generateGenesisCommand)
370+
commands ++= Seq(compilePR, checkPR, buildReleaseArtifacts, generateGenesisCommand)

‎node-it/build.sbt‎

Lines changed: 23 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -1,15 +1,31 @@
1-
enablePlugins(IntegrationTestsPlugin, sbtdocker.DockerPlugin)
1+
import sbt.*
2+
import sbt.Keys.*
3+
4+
import scala.sys.process.*
5+
6+
enablePlugins(IntegrationTestsPlugin)
27

38
description := "NODE integration tests"
49
libraryDependencies ++= Dependencies.it
510

6-
inTask(docker)(
7-
Seq(
8-
imageNames := Seq(ImageName("com.wavesplatform/node-it")),
9-
dockerfile := NativeDockerfile(baseDirectory.value.getParentFile / "docker" / "Dockerfile"),
10-
buildOptions := BuildOptions()
11+
val docker = taskKey[Unit]("Build docker image for integration tests")
12+
docker := {
13+
val log = streams.value.log
14+
15+
val cwd = baseDirectory.value.getParentFile / "docker"
16+
val image = "com.wavesplatform/node-it:latest"
17+
18+
val cmd = Seq("docker", "build", "-t", image, ".")
19+
log.info(s"Running `${cmd.mkString(" ")}` from $cwd")
20+
21+
val processLogger = ProcessLogger(
22+
(out: String) => log.info(out),
23+
(err: String) => log.info(err) // Redirect STDERR to info
1124
)
12-
)
25+
26+
val exit = Process(cmd, cwd).!(processLogger)
27+
if (exit != 0) sys.error(s"Docker build failed with exit code $exit")
28+
}
1329

1430
val buildTarballsForDocker = taskKey[Unit]("build all packages")
1531

‎node-it/src/test/scala/com/wavesplatform/it/BaseTargetChecker.scala‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -38,7 +38,7 @@ object BaseTargetChecker {
3838
blockchainUpdater.isFeatureActivated(BlockchainFeatures.LightNode)
3939
)
4040
.explicitGet()
41-
blockchainUpdater.processBlock(genesisBlock, genesisBlock.header.generationSignature, snapshot = None, generatorBalances = Seq.empty)
41+
blockchainUpdater.processBlock(genesisBlock, genesisBlock.header.generationSignature, snapshot = None, generatorSet = Seq.empty)
4242

4343
NodeConfigs.Default.map(_.withFallback(sharedConfig)).collect {
4444
case cfg if ConfigSource.fromConfig(cfg).at("waves.miner.enable").loadOrThrow[Boolean] =>

‎node-it/src/test/scala/com/wavesplatform/it/Docker.scala‎

Lines changed: 21 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -504,6 +504,14 @@ class Docker(
504504
log.info(s"New ports: ${ports.toString}")
505505
client.restartContainer(id, 10)
506506

507+
node.nodeInfo = Iterator
508+
.continually {
509+
Thread.sleep(1.second.toMillis)
510+
getNodeInfo(node.containerId, node.settings)
511+
}
512+
.dropWhile(_.ports.isEmpty)
513+
.next()
514+
507515
node.nodeInfo = getNodeInfo(node.containerId, node.settings)
508516
Await.result(
509517
node.waitForStartup().flatMap(_ => connectToAll(node)),
@@ -573,7 +581,19 @@ object Docker {
573581
val initialWavesAmount: Long = configTemplate.getLong("waves.blockchain.custom.genesis.initial-balance")
574582

575583
def genesisOverride(featuresConfig: Option[Config] = None): Config = {
576-
val genesisTs: Long = System.currentTimeMillis()
584+
// Starting a node and applying the genesis block takes a non-negligible amount of time. If we do not introduce an offset,
585+
// the system will treat the genesis block as if it was created in the past. In CI runs, this time gap can reach up
586+
// to 30 seconds.
587+
//
588+
// Block mining starts immediately after genesis is applied. As a result, there may be less time available for a
589+
// second block than some tests require (for example, to populate it with transactions).
590+
//
591+
// If the genesis block timestamp is slightly in the future, it will still be accepted. The only side effect is a
592+
// delayed start of mining.
593+
//
594+
// The chosen offset represents a compromise between realistic timing and test stability.
595+
val offsetMs = 12_000
596+
val genesisTs: Long = System.currentTimeMillis() + offsetMs
577597

578598
val timestampOverrides = parseString(s"""waves.blockchain.custom.genesis {
579599
| timestamp = $genesisTs

‎node-it/src/test/scala/com/wavesplatform/it/async/MicroblocksGenerationSuite.scala‎

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -2,12 +2,13 @@ package com.wavesplatform.it.async
22

33
import com.typesafe.config.{Config, ConfigFactory}
44
import com.wavesplatform.it.api.AsyncHttpApi.*
5-
import com.wavesplatform.it.{BaseFreeSpec, NodeConfigs, TransferSending}
5+
import com.wavesplatform.it.{BaseFreeSpec, LoadTest, NodeConfigs, TransferSending}
66
import com.wavesplatform.state.Height
77

88
import scala.concurrent.Await.result
99
import scala.concurrent.duration.*
1010

11+
@LoadTest
1112
class MicroblocksGenerationSuite extends BaseFreeSpec with TransferSending {
1213
import MicroblocksGenerationSuite.*
1314

‎node-it/src/test/scala/com/wavesplatform/it/sync/finalization/OneNodeFinalizationTestSuite.scala‎

Lines changed: 18 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -8,6 +8,7 @@ import com.wavesplatform.it.api.SyncHttpApi.*
88
import com.wavesplatform.it.{BaseFreeSpec, NodeConfigs}
99
import com.wavesplatform.state.Height
1010
import com.wavesplatform.test.NumericExt
11+
import org.apache.pekko.http.scaladsl.model.StatusCodes
1112
import org.scalatest.OptionValues
1213

1314
import scala.concurrent.duration.DurationInt
@@ -65,6 +66,18 @@ class OneNodeFinalizationTestSuite extends BaseFreeSpec with OptionValues {
6566
var finalizedHeight1 = node.finalizedHeight
6667
val waitingFinalizedHeight = finalizedHeight1 + 2
6768

69+
withClue("Finalized height is unknown: ") {
70+
try node.finalizedHeightAt(node.height)
71+
catch {
72+
case ApiCallException(e: UnexpectedStatusCodeException) => e.statusCode shouldBe StatusCodes.NotFound.intValue
73+
}
74+
75+
try node.finalizedHeightAt(node.height + 10)
76+
catch {
77+
case ApiCallException(e: UnexpectedStatusCodeException) => e.statusCode shouldBe StatusCodes.NotFound.intValue
78+
}
79+
}
80+
6881
var done = false
6982
while (!done && deadline.hasTimeLeft()) {
7083
val currHeight = node.height
@@ -98,20 +111,15 @@ class OneNodeFinalizationTestSuite extends BaseFreeSpec with OptionValues {
98111
step("Finalized block header and height checks")
99112
val finalizedBlock1 = node.finalizedBlockHeader()
100113
finalizedBlock1.height should be >= finalizedHeight1
101-
102-
val finalizedHeight2 = node.finalizedHeightAt(node.height)
103-
finalizedHeight2 should be >= finalizedHeight1
104-
105-
val finalizedHeightBefore1 = node.finalizedHeightAt(finalizedBlock1.height)
106-
finalizedHeightBefore1 should be < finalizedHeight1
114+
node.finalizedHeightAt(finalizedBlock1.height) should be <= finalizedBlock1.height
107115

108116
step("Finalization voting in a block header")
109-
val blockHeader = node.blockHeaderAt(node.height - 1)
110-
val finalizationVoting = blockHeader.finalizationVoting.value
117+
val votingBlockHeader = node.blockHeaderAt(finalizedHeight1 + 1)
118+
val finalizationVoting = votingBlockHeader.finalizationVoting.value
111119

112-
val generators: Seq[(data: GeneratorsResponse.Entry, index: Int)] = node.generators(blockHeader.height).zipWithIndex
120+
val generators: Seq[(data: GeneratorsResponse.Entry, index: Int)] = node.generators(votingBlockHeader.height).zipWithIndex
113121

114-
val minerEndorser = generators.find { g => g.data.address == blockHeader.generator }.value
122+
val minerEndorser = generators.find { g => g.data.address == votingBlockHeader.generator }.value
115123

116124
withClue(s"endorsers=[${finalizationVoting.endorserIndexes.mkString(", ")}], miner=${minerEndorser.index}: ") {
117125
finalizationVoting.endorserIndexes should not contain minerEndorser.index

‎node-it/src/test/scala/com/wavesplatform/it/sync/finalization/TwoNodesFinalizationTestSuite.scala‎

Lines changed: 6 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -15,8 +15,8 @@ import scala.concurrent.duration.DurationInt
1515
class TwoNodesFinalizationTestSuite extends BaseFreeSpec with OptionValues {
1616
override protected def nodeConfigs: Seq[Config] =
1717
NodeConfigs.newBuilder
18-
.overrideBase(_.quorum(0))
1918
.overrideBase(_.preactivatedFeatures((BlockchainFeatures.DeterministicFinality.id, Height(0))))
19+
.overrideBase(_.raw("waves.miner.minimal-block-generation-offset = 10s"))
2020
.withDefault(2)
2121
.buildNonConflicting()
2222

@@ -34,9 +34,12 @@ class TwoNodesFinalizationTestSuite extends BaseFreeSpec with OptionValues {
3434
step("Commit to generation")
3535
val commitTxn1 = node1.sign(CommitToGenerationRequest(sender = Some(miner1Addr)))
3636
val commitTxn2 = node2.sign(CommitToGenerationRequest(sender = Some(miner2Addr)))
37-
node1.broadcastRequest(commitTxn1)
38-
node1.broadcastRequest(commitTxn2)
37+
Seq(node1, node2).foreach { node =>
38+
node.broadcastRequest(commitTxn1)
39+
node.broadcastRequest(commitTxn2)
40+
}
3941
node1.waitForGenerationPeriod(period1)
42+
node2.waitForGenerationPeriod(period1)
4043

4144
step("Generators")
4245
isolated {

0 commit comments

Comments
 (0)
⚡