[fix] Real time updates on visualizer view by Dhanus3133 · Pull Request #257 · openwisp/openwisp-network-topology

Dhanus3133 · 2025-07-24T16:53:58Z

Closes #250

Closes #251

nemesifier

Probably you need to modify the CI or test requirements to install the channels branch which has the patch for the live server?

nemesifier · 2025-07-24T17:33:22Z

        "ENGINE": "openwisp_utils.db.backends.spatialite",
        "NAME": os.path.join(BASE_DIR, "openwisp_network_topology.db"),
+        "TEST": {
+            "NAME": os.path.join(BASE_DIR, "test_openwisp_network_topology.db"),


Is this really needed?

Yeah, ChannelLiveServerTestCase can't run with an in-memory database.

Dhanus3133 · 2025-07-24T17:55:40Z

When running tests in parallel with Django versions < 5, I encounter an error due to ChannelLiveServerTestCase:

ERROR: test_real_time_link_status_update (openwisp_network_topology.tests.test_realtime.TestRealTime)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/opt/hostedtoolcache/Python/3.10.18/x64/lib/python3.10/site-packages/django/test/testcases.py", line 407, in _setup_and_call
    self._pre_setup()
  File "/opt/hostedtoolcache/Python/3.10.18/x64/lib/python3.10/site-packages/channels/testing/live.py", line 61, in _pre_setup
    self._server_process.start()
  File "/opt/hostedtoolcache/Python/3.10.18/x64/lib/python3.10/multiprocessing/process.py", line 118, in start
    assert not _current_process._config.get('daemon'), \
AssertionError: daemonic processes are not allowed to have children

https://github.com/openwisp/openwisp-network-topology/actions/runs/16502989458/job/46666342034?pr=257#step:10:83

nemesifier

We're almost there with this one.

The following test is too flaky in my environment:

[Retry] Retrying "openwisp_network_topology.tests.test_realtime.TestRealTime.test_node_link_update", attempt 1/5. 
--------------------------------------------------------------------------------
--------------------------------------------------------------------------------
[Retry] Retrying "openwisp_network_topology.tests.test_realtime.TestRealTime.test_node_link_update", attempt 2/5. 
--------------------------------------------------------------------------------
--------------------------------------------------------------------------------
[Retry] Retrying "openwisp_network_topology.tests.test_realtime.TestRealTime.test_node_link_update", attempt 3/5. 
--------------------------------------------------------------------------------
--------------------------------------------------------------------------------
[Retry] Retrying "openwisp_network_topology.tests.test_realtime.TestRealTime.test_node_link_update", attempt 4/5. 
--------------------------------------------------------------------------------
--------------------------------------------------------------------------------
[Retry] Retrying "openwisp_network_topology.tests.test_realtime.TestRealTime.test_node_link_update", attempt 5/5. 
--------------------------------------------------------------------------------

I disabled the retry logic in opernwisp-utils to see the stacktrace:

ERROR: test_node_link_update (openwisp_network_topology.tests.test_realtime.TestRealTime)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/nemesis/.virtualenvs/openwisp2/lib/python3.10/site-packages/asgiref/sync.py", line 262, in __call__
    return call_result.result()
  File "/usr/lib/python3.10/concurrent/futures/_base.py", line 451, in result
    return self.__get_result()
  File "/usr/lib/python3.10/concurrent/futures/_base.py", line 403, in __get_result
    raise self._exception
  File "/home/nemesis/.virtualenvs/openwisp2/lib/python3.10/site-packages/asgiref/sync.py", line 302, in main_wrap
    result = await awaitable
  File "/home/nemesis/Code/openwisp/openwisp-network-topology/openwisp_network_topology/tests/test_realtime.py", line 190, in test_node_link_update
    len(self.web_driver.execute_script("return graph.data;")["links"]),
  File "/home/nemesis/.virtualenvs/openwisp2/lib/python3.10/site-packages/selenium/webdriver/remote/webdriver.py", line 528, in execute_script
    return self.execute(command, {"script": script, "args": converted_args})["value"]
  File "/home/nemesis/.virtualenvs/openwisp2/lib/python3.10/site-packages/selenium/webdriver/remote/webdriver.py", line 429, in execute
    self.error_handler.check_response(response)
  File "/home/nemesis/.virtualenvs/openwisp2/lib/python3.10/site-packages/selenium/webdriver/remote/errorhandler.py", line 232, in check_response
    raise exception_class(message, screen, stacktrace)
selenium.common.exceptions.JavascriptException: Message: ReferenceError: graph is not defined
Stacktrace:
@moz-nullprincipal:{5659b005-1c36-40dd-a1c2-c1026854d042}:2:7
@moz-nullprincipal:{5659b005-1c36-40dd-a1c2-c1026854d042}:3:8

This is what I see if I use to stop the testing right before the failing line (with ipdb):

An empty tab, which of course doesn't yield the desired result. Can you please change the test to fix this issue? I suggest using ipdb to run the code line by line in this test and find a different way to implement it that doesn't cause flakyness.

coveralls · 2025-07-31T16:34:43Z

coverage: 98.987% (+0.7%) from 98.29%
when pulling a22fafc on bug/visualizer-view-realtime-updates
into 920624c on master.

nemesifier

Is there a test for the non-admin visualizer though (which #250 is all about)?

Too flaky

Dhanus3133 · 2025-08-03T17:35:13Z

LGTM!

pandafy

@Dhanus3133 While debugging some issues in openwisp/openwisp-firmware-upgrader#320, I had some doubts.

pandafy · 2025-08-18T17:59:35Z

+    node_model = Node
+    link_model = Link
+    topology_model = Topology
+    application = import_string(getattr(settings, "ASGI_APPLICATION"))


Why do we need this? This is imported when the class is loaded. This may happen before ready for all app config is executed.

It wasn't imported. I tried it.

openwisp-network-topology/openwisp_network_topology/tests/utils.py

Lines 20 to 37 in 20f8800

class CreateGraphObjectsMixin(object):

def _create_topology(self, **kwargs):

options = dict(

label="TestNetwork",

parser="netdiff.OlsrParser",

strategy="fetch",

url="http://127.0.0.1:9090",

protocol="OLSR",

version="0.8",

metric="ETX",

created="2017-07-10T20:02:52.483Z",

modified="2015-07-14T20:02:52.483Z",

)

options.update(kwargs)

t = self.topology_model(**options)

t.full_clean()

t.save()

return t

What did you try?

My question was regarding application = import_string(getattr(settings, "ASGI_APPLICATION"))

In channels, I see the following code:

https://github.com/django/channels/blob/a1352286ab4aa54ba5de7acdfdce44dcc73fecf5/channels/testing/live.py#L67-L70

https://github.com/django/channels/blob/a1352286ab4aa54ba5de7acdfdce44dcc73fecf5/channels/testing/live.py#L14-L19

https://github.com/django/channels/blob/a1352286ab4aa54ba5de7acdfdce44dcc73fecf5/channels/routing.py#L15-L33

So, the test class will automatically get the application from the Django settings.

Oh, I thought you were talking about models since GitHub highlighted those too. Yeah, it seems like we could have used it as well. I used this as a reference after seeing it in another test:

openwisp-network-topology/openwisp_network_topology/tests/test_websockets.py

Line 28 in 0685cae

application = import_string(getattr(settings, "ASGI_APPLICATION"))

pandafy · 2025-08-18T18:00:27Z

+        self.org = org
+
+    async def _prepare(self, admin=True):
+        communicator = await self._get_communicator(self.admin_client, self.topology.pk)


What is the use of communicator in selenium tests? Aren't we making sending/receiving data using the browser directly?

To verify WebSocket changes. No, we are updating directly using queries.

Aren't we ensuring that the UI updates after receiving data from the websocket?

Aren't we ensuring that the UI updates after receiving data from the websocket?

Dhanus here verified both:

data is received as expected

the UI updates as expected, with some limitations here due to the use of canvas but we're checking the JS data structures have updated as expected

I think it's a solid approach.

Dhanus3133 added 2 commits July 24, 2025 22:21

[fix] Real time updates on visualizer view (#250)

32b5c80

Closes #250

[tests] Add browser test for realtime updates (#251)

1839f79

Closes #251

nemesifier reviewed Jul 24, 2025

View reviewed changes

nemesifier assigned Dhanus3133 Jul 30, 2025

[tests] Optimizations for non-parallel tests

425b480

nemesifier force-pushed the bug/visualizer-view-realtime-updates branch from da4f489 to 425b480 Compare July 30, 2025 15:34

nemesifier requested changes Jul 30, 2025

View reviewed changes

nemesifier marked this pull request as ready for review July 30, 2025 15:38

nemesifier mentioned this pull request Jul 30, 2025

[feature:ui] Show upgrade progress for single upgrade operations in real time openwisp/openwisp-firmware-upgrader#320

Merged

4 tasks

Dhanus3133 and others added 3 commits July 31, 2025 15:45

[tests] Use Chromium instead of firefox

6ac4f7e

[tests] Switched channels memory to redis and other fixes

35048b2

[chores] Added missing snooze and comment

f4a4d6a

[tests] Exec realtime tests first, increase snooze time

a22fafc

nemesifier added the bug label Jul 31, 2025

nemesifier reviewed Jul 31, 2025

View reviewed changes

Dhanus3133 and others added 7 commits August 1, 2025 19:12

[tests] Add test for non admin visualizer

26bb98e

[chores] Various improvements to be discussed

9348856

[chore] Set maxDiff to None

72cecfa

[chores] Do not check browser logs in test_non_admin_visualizer

9daa329

Too flaky

[chores] Tweaks

25a6ff4

[chores] Too flaky..

f8838c9

[chores] Ignore library errors

9b4558d

nemesifier approved these changes Aug 2, 2025

View reviewed changes

Merge branch 'master' into bug/visualizer-view-realtime-updates

5d6e9a6

nemesifier approved these changes Aug 4, 2025

View reviewed changes

nemesifier merged commit f8bb68c into master Aug 4, 2025
13 checks passed

nemesifier deleted the bug/visualizer-view-realtime-updates branch August 4, 2025 17:36

pandafy reviewed Aug 18, 2025

View reviewed changes

	class CreateGraphObjectsMixin(object):
	def _create_topology(self, **kwargs):
	options = dict(
	label="TestNetwork",
	parser="netdiff.OlsrParser",
	strategy="fetch",
	url="http://127.0.0.1:9090",
	protocol="OLSR",
	version="0.8",
	metric="ETX",
	created="2017-07-10T20:02:52.483Z",
	modified="2015-07-14T20:02:52.483Z",
	)
	options.update(kwargs)
	t = self.topology_model(**options)
	t.full_clean()
	t.save()
	return t

Uh oh!

Conversation

Dhanus3133 commented Jul 24, 2025

Uh oh!

nemesifier left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Dhanus3133 commented Jul 24, 2025

Uh oh!

nemesifier left a comment

Choose a reason for hiding this comment

Uh oh!

coveralls commented Jul 31, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

nemesifier left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Dhanus3133 commented Aug 3, 2025

Uh oh!

Uh oh!

pandafy left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

coveralls commented Jul 31, 2025 •

edited

Loading

nemesifier left a comment •

edited

Loading