Architecture Note¶
Purpose¶
This note summarises the current architecture of VisIVO Visual Analytics after the full client/backend integration, async viewer additions, authentication layer, and diagnostic infrastructure.
It describes what is already true in the codebase today, which boundaries are now stable, and what remains deferred.
High-Level Topology¶
┌─────────────────────────────────────────────────┐
│ Qt/VTK Desktop Client │
│ │
│ main() ──► StartupDialog │
│ │ BackendLauncher (process mgmt) │
│ ▼ │
│ MainWindow ──► DataHubWidget │
│ │ │
│ ├──► vtkWindowCube (local/remote) │
│ ├──► vtkWindowImage (local/remote) │
│ ├──► vtkWindowCatalogue3D (remote-only) │
│ ├──► vtkWindowVbt (remote-only) │
│ ├──► vtkWindowVbtVolume (remote-only) │
│ ├──► HiPSWindow (remote-only) │
│ └──► RemoteMomentWindow (remote-only) │
│ │
│ BackendClient ── Qt Network ──► FastAPI backend │
│ AuthWrapper ── OIDC/PKCE ──► identity server │
│ DiagnosticsManager (singleton, in-process log) │
└─────────────────────────────────────────────────┘
MainWindow is created only after StartupDialog accepts — i.e., only after the backend is reachable and a token is available (or explicitly skipped). The backend process is owned by BackendLauncher in main() and outlives all windows.
Build Modules¶
Client GUI (src/gui/, src/auth/, src/)¶
Qt widget layer and UI orchestration.
Contents:
StartupDialog— pre-MainWindowdialog; drives Backend → Auth → Ready sequenceMainWindow— top-level shell, menus, action routingDataHubWidget— landing page; health check, file browser, session startvtkWindowCube— FITS cube viewer (local preview + remote full, moment, PV, noise)vtkWindowImage— FITS image viewer (local + remote multi-layer); “Add New FITS File” usesRemoteFileBrowserDialogvtkWindowCatalogue3D— 3-D catalogue viewer (remote CSV/backend)vtkWindowVbt— VBT point-cloud viewer (remote)vtkWindowVbtVolume— VBT volume viewer (remote)HiPSWindow/HiPSViewportWidget— HiPS sky viewer with overlayRemoteMomentWindow— standalone moment map viewer (async task-based)RemoteFileBrowserDialog— file-picker backed by/v1/files/list; reused for image layersNoiseRegionDialog— region selector for noise computationProfileWidget,PvDiagramWidget— profile/PV plot widgetsDiagnosticsWindow— live table view ofDiagnosticsManagerentriesAuthWrapper,OIDCAuthorizationCodeFlow— OIDC PKCE authLUTCustomizerDialog,SettingsDialog,AboutDialogCatalogue3DParser,Catalogue3DTableModel,CatalogueTableModelVisivoTheme— application-wide style sheet
Third-party bundled¶
libwcs— WCS coordinate conversion (C, statically linked)qcustomplot— 1-D profile and PV plots
Backend (FastAPI, Python)¶
The backend process (backend/app/main.py) runs as a local or remote server.
The client talks to it exclusively through BackendClient (REST/HTTP, bearer-token auth).
Endpoint groups¶
Tag |
Endpoints |
Notes |
|---|---|---|
|
|
startup health check, session stats, per-session dataset enumeration (used by cross-dataset tools like Spectral Stacking) |
|
|
backend-side filesystem browser |
|
|
open FITS/dataset, returns |
|
|
CSV catalogue; paginated query via |
|
|
VBT point table; paginated query |
|
|
cube data slices, PV diagram, noise stats; |
|
|
synchronous moment / isosurface computation; |
|
|
Workspace Exports lifecycle: enumerate, stream-download, remove. All operations are path-traversal-safe; absolute paths and |
|
|
async task queue; client polls for completion |
|
|
2-D image export/preview |
|
|
redshift → comoving distance (astropy) |
|
|
HiPS sky survey tiles + source overlay |
|
|
astronomical name → sky coordinates |
|
|
SAMP messaging + file sharing; no auth dependency so the local SAMP hub can reach it directly |
|
|
per-pixel FWHM + EW maps (S-02), polynomial / median baseline subtraction (S-03), spectral stacking (S-04) |
Backend session model¶
POST /v1/datasets/openreturns asession_id; the client echoes it asX-Visivo-Sessionin all subsequent requests for that datasetSessions are tracked server-side;
GET /v1/sessionslists active sessionsAuth: every request carries
X-Visivo-Token(static bearer token); separate OIDC flow covers the VLKB identity service
BackendClient¶
Single synchronous REST client (src/app/BackendClient.h/cpp).
Key design decisions:
all methods are blocking — called from
QtConcurrent::runworker threads, never from the UI threadsession management:
setSessionId()/sessionId(); echoed automatically asX-Visivo-Sessiontoken resolution order: (1) explicit
setToken(), (2)~/.visivo_tokenfile written by the backend at startupstatic parse helpers exposed for unit testing:
parseMomentResultObject,parsePvResultObject,parseNoiseResultObjecttimeout: per-request via
requestTimeoutFor(); long-running endpoints (subvolume, full-res cube) get extended timeouts
Result structs (all in BackendClient.h):
Struct |
Produced by |
|---|---|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
SAMP send / send-fits / send-catalogue |
|
SAMP import-url / upload-file |
|
|
|
|
Async Patterns¶
All viewers follow the same pattern:
The UI thread captures needed state and launches a
QtConcurrent::runworkerThe worker uses its own
BackendClientinstance (no shared state)A
QFutureWatcherfiresfinished()on the UI threadThe UI thread reads the result struct and applies it (scene rebuild, LUT update, etc.)
Controls are disabled during the operation and re-enabled on completion
Active async paths¶
Viewer |
Watcher(s) |
Triggered by |
|---|---|---|
|
|
open, slice change, threshold change, moment/PV requests |
|
|
local “Add layer”, remote image preview, remote full-res upgrade |
|
|
Apply filter, Load more, cosmology model change |
|
|
Apply filter, Load more |
The cube preview→full-res path uses worker-side sanitization and a deferred-Render
pattern to keep the UI thread reactive during the swap — see
docs/async-patterns.md (sections Worker-side data preparation and
Deferred render after data swap) for the details.
Pagination (large datasets)¶
Both vtkWindowCatalogue3D and vtkWindowVbt support incremental pagination for datasets larger than the default page size (50 000 rows):
m_pageSize = 50000,m_currentOffset,m_totalCountapplyFilter()always resets offset to 0 and fetches the first pageA “Load more (N remaining)” button appears in the filter panel when
offset < totalCountClicking it triggers
loadMoreEntries()viam_loadMoreWatcherCatalogue3D: new entries are appended to
this->entries; the full VTK scene is rebuiltVBT: new column vectors are appended to
this->subsetResult.columns; the point cloud is rebuilt
Task-based async (moment, PV)¶
For long-running operations the backend supports a task queue:
createMomentTask()/createPvTask()→ returns atask_idwaitForTaskCompletion()pollsGET /v1/tasks/{id}with exponential back-offRemoteMomentWindowuses this path exclusively
Authentication¶
Backend token¶
Static bearer token, resolved by BackendClient from:
Explicit
setToken()call (from Settings)~/.visivo_tokenfile written by the backend process at startup
VLKB OIDC¶
AuthWrapper wraps OIDCAuthorizationCodeFlow (PKCE) for the VLKB identity service.
AuthWrapper::grant(AuthService::VLKB)launches the browser-based flowHttpServerReplyHandlercaptures the redirect callback onlocalhostVisIVOUrlSchemeHandlerhandles the customvisivo://URL scheme on macOSTokens are stored in-process;
logout()clears them
Diagnostics¶
DiagnosticsManager (singleton, src/app/) is an in-process structured event log.
publish(level, category, source, message, datasetId, sessionId, operationTag)— called from any threadCategories:
Scientific,Client,Backend,Task,WCS,Remote,Rendering,PerformanceDiagnosticsModelexposes the log as aQAbstractTableModelforDiagnosticsWindowDiagnosticsWindow::showSingleton()returns the one process-wide window, lazily created on first call. Used byMainWindow::openDiagnosticsWindow(), the Command Palette entry “Open Diagnostics Window”, and the replicated View → Diagnostics action in child windows (vtkWindowCube,vtkWindowImage). All entry points share the same model — no duplicate panels.
Command Palette (⌘K)¶
CommandPalette (src/gui/) is a search-driven action launcher owned by
MainWindow. Ctrl+K is registered as an ApplicationShortcut so it fires from
any window. Child windows also expose an explicit View → Command Palette… entry
that routes through QMetaObject::invokeMethod(mainWindow, "openCommandPalette", QueuedConnection) — MainWindow::openCommandPalette is declared Q_INVOKABLE so
child windows do not need to link against MainWindow to trigger it.
Cube viewer (vtkWindowCube) — interactive extras¶
In addition to the volume / isosurface / slice / moment / PV / noise pipelines,
the cube viewer exposes a set of configurable interaction extras driven from
the View menu and mirrored in the sidebar. The dedicated “Cube Extras” tab
was removed; its widgets were redistributed into the 3-D View Settings and
2-D View Settings pages where they semantically belong. All toggles still
use a checkable QAction as the single source of truth — sidebar buttons
bind to it via QToolButton::setDefaultAction(), which gives Qt bidirectional
state sync for free (no manual connect pairs needed for the boolean toggles).
Feature |
QAction / group |
Menu |
Sidebar widget |
|---|---|---|---|
Cutting plane visibility |
|
View → Show Cutting Plane |
3-D View Settings → CUTTING PLANE → “Show cutting plane” toggle button (full-width, |
Cutting plane opacity (0–100%) |
|
View → Cutting Plane Opacity → … |
3-D View Settings → CUTTING PLANE → continuous slider with read-only QLineEdit value below (matches RENDERING THRESHOLD layout) |
3D WCS axes ( |
|
View → Show 3D WCS Axes |
3-D View Settings → 3D REFERENCE → “Show 3D WCS axes” toggle button |
Slice animation |
|
View → Play Slices / Animation Speed / Animation Mode |
2-D View Settings → SLICE ANIMATION → “▶ Play” button + FPS combo (preset 2 / 5 / 10 / 15 / 30, no free-form entry) + Mode combo (Loop / Bounce / Stop at End) |
3D pick on plane click |
|
View → Pick Spectrum on Plane Click |
3-D View Settings → 3D INTERACTION → “Pick spectrum on plane click” toggle button |
Slice contours overlay |
|
— |
2-D View Settings → CONTOURS → “Show Contours” toggle button (drives the hidden checkbox; existing |
2-D view mode |
menu actions |
View → Slice / Moment Map |
2-D View Settings → inline |
Implementation notes:
Cutting plane is a textured
vtkActor. The plane source feeds avtkPolyDataMapper; avtkTextureis attached to the actor with inputsliceColors->GetOutputPort()(the samevtkImageMapToColorspipeline that drives the 2D slice view). Slice tile updates propagate to the 3D plane automatically via the VTK pipeline. The texture is wired in the constructor aftersetupSliceRenderer()so thatsliceColorshas a valid input by the time the first 3D Render runs; otherwise the volume scene would not draw at all.Animation timer is a
QTimermember;setSliceAnimationActive(true)starts it with1000 / fpsinterval,advanceSliceAnimation()drivesspinSliceaccording to mode (Loop / Bounce / Stop at End). The timer is stopped incloseEvent().3D spectral pick installs a
vtkCallbackCommandon the cube interactor’sLeftButtonReleaseEvent(priority 1.0). On pick:vtkPropPickerresolves the prop under the cursor; if it isremoteCuttingPlaneActor, the world XY is mapped to voxel indices and fed into the existing probe pipeline (updateProbePlot()→ProfileWidget).probeModeActiveis flipped on directly (skippingsetProbeModeActiveso the 2D cursor and region actions are left untouched). The observer is removed incloseEvent()and when the toggle is unchecked.3D WCS axes wraps a
vtkCubeAxesActorwith titles fromremoteAxisTitle(0..2)and ranges fromremoteVoxelToWcson the cube bounds;applyCubeOpenResult()refreshes the actor whenever the cube bounds change (preview → full-res, ROI switch).
Beam indicator ellipse (2-D slice view)¶
The beam indicator is a filled ellipse rendered in the bottom-left corner
of the 2-D slice renderer. It visualises the synthesised beam reported by
the FITS header keywords BMAJ, BMIN, and BPA.
Implementation notes:
Backend:
geometry_metadata()inbackend/app/fits_dataset.pyreadsBMAJ,BMIN, andBPAfrom the primary header and returns them (in degrees) asbeam_major,beam_minor,beam_pa. TheOpenDatasetResponseschema (backend/app/schemas.py) carries them as optional floats; the client receives them viaBackendOpenDatasetResultfieldsbeamMajorDeg,beamMinorDeg,beamPaDeg.Frontend:
vtkWindowCube::setBeamInfo()builds an ellipse from parametric points (cos/sin sampled at ~64 steps), converts angular sizes to pixels via|CDELT1|, and creates a filledvtkActor2Dwith white colour and semi-transparent opacity. The actor is added to the 2-D slice renderer at a fixed position in the bottom-left corner (viewport-relative coordinates).If
BMAJorBMINare absent (i.e. the optional fields are unset), the ellipse actor is not created or is set invisible — no fallback drawing occurs.
Spectral smoothing (ProfileWidget)¶
The ProfileWidget spectrum header exposes a Smooth: QComboBox that
applies a 1-D convolution kernel to the displayed profile.
Implementation notes:
Kernels available: None, Hanning
[0.25, 0.5, 0.25], Boxcar 3/5/7, Gaussian σ=1, Gaussian σ=2. The kernel array is applied via aconvolve1Dhelper function.The convolution is NaN-safe: NaN samples are excluded from the weighted sum and the normalisation factor is adjusted to compensate, so NaN values do not propagate into neighbouring channels.
The stats bar (N, Min, Max, Mean, RMS, ∫) is recomputed on the smoothed data vector, giving the user immediate quantitative feedback on the effect of the kernel.
Smoothing is active during live probe hover: every
updateProbePlot()call re-applies the selected kernel before plotting, so changing kernels while hovering is responsive.CSV export always writes the raw (unsmoothed) data to preserve scientific provenance.
Line identification overlay (ProfileWidget)¶
Load Lines… and Clear Lines buttons in the ProfileWidget header
allow the user to overlay expected spectral-line positions.
Implementation notes:
The CSV parser accepts comma- or tab-separated files with two columns (
frequency,label). Lines beginning with#are skipped as comments. No header row is required.Each loaded line is rendered as a
QCPItemLine(vertical, dashed, amber pen) spanning the full Y range of the plot. The label is rendered as aQCPItemTextpositioned at the line’s X coordinate with a viewport-ratio Y coordinate (fixed fraction of the plot height, e.g. 0.85) so that labels stay readable regardless of zoom level. Labels are rotated 90°.Clear Lines removes all
QCPItemLine+QCPItemTextitems that belong to the line-ID overlay set and replots.No unit conversion is performed: frequencies in the file must match the plot’s current X-axis unit.
Optional: VR (OpenXR) offload¶
The cube viewer can hand off the current vtkVolume (with its live LUT /
opacity TF / threshold) to a head-mounted display via a second OpenXR-backed
render window — same actor, same mapper, so every desktop-side parameter
change is reflected in the headset on the next frame without any IPC.
Build is opt-in and does not affect the default macOS / Linux / Windows build. Three states:
|
VTK has |
Result |
|---|---|---|
|
irrelevant |
Identical to today: Tools → Open in VR still appears in the menu but is disabled with an explanatory tooltip. |
|
no |
CMake prints a |
|
yes |
|
Implementation notes:
src/gui/CubeVRController.{h,cpp}— PIMPL controller. The header is always compilable; OpenXR includes live behind#ifdef VISIVO_HAS_VRin the.cpponly.CubeVRController::isCompiledIn()— compile-time flag.CubeVRController::isRuntimeAvailable()— lazy runtime probe (vtkOpenXRRenderWindow::Initialize()), result cached for the session.CubeVRController::open(renderer, volume)— shares the desktop volume actor with a secondvtkOpenXRRenderWindow. Currently blocks the UI thread for the duration of the session (matches the user mental model “I’m in VR until I take the headset off”); a future iteration can move this to aQThreadwith a mutex on the shared mapper.macOS is not a supported VR target (Apple removed SteamVR support in 2020; no Vision Pro / OpenXR runtime). The build flag is honoured anyway — it just falls into the “no runtime” branch.
To enable end-to-end on Windows / Linux:
Rebuild VTK with
-DVTK_MODULE_ENABLE_VTK_RenderingOpenXR=YES. Requires the Khronos OpenXR loader headers (Linux:libopenxr-loader1-dev/ equivalent; Windows: Khronos OpenXR SDK).Install an OpenXR runtime (SteamVR, Oculus, WMR, Monado, …) — pick the one bundled with your HMD vendor.
Configure VisIVO with
cmake -DVISIVO_ENABLE_VR=ON ….Launch, open a cube, Tools → Open in VR.
QAction creation ordering¶
QActions referenced by the sidebar’s redistributed Extras sections
(Cutting Plane / 3D Reference / 3D Interaction / Slice Animation) must be
created before setupSidebar() is called — otherwise the sidebar
dereferences null pointers and the window segfaults. In the constructor:
ui->setupUi(this);
setupViewerToolbar();
// 1. Pre-create QActions + QActionGroups used by sidebar
actionShowCuttingPlane = new QAction(…); …
cuttingPlaneOpacityGroup = new QActionGroup(this); …
// 2. Build sidebar (binds widgets to the pre-created actions)
setupSidebar();
// 3. Later: attach tooltips, populate menus, connect signal handlers
Workspace Exports (persistent FITS artefacts)¶
Cube viewer “Export … as FITS” actions deposit their products in a persistent workspace directory on the backend host, not in temp. Files survive across sessions and are browsable / re-openable from the client without the user having to remember a path.
Config & layout
Directory:
$VISIVO_EXPORTS_DIRenv var, default~/.visivo/exports/. Created on backend startup (app.dependencies._EXPORTS_DIR).Flat namespace — no per-session subdirs. Filename collisions are resolved by
make_workspace_path()with a numeric suffix:cube.fits→cube_1.fits→cube_2.fits→ …Empty / dot-only / traversal-prefixed basenames fall back to
export.fitsso the workspace can never be escaped.
Backend producers
Endpoint |
Writer |
Produces |
|---|---|---|
|
|
Cropped 3-D FITS (WCS preserved, CRPIX shifted). Also registered as a new session dataset so the client can immediately open it without re-uploading. |
|
|
A single channel of a cube written as a standalone 2-D FITS (NAXIS=2, spectral axis dropped via |
|
|
2-D moment FITS (celestial WCS + BUNIT). Not registered as a session dataset (it’s an image, not a cube). |
Both accept an optional output_basename; empty defaults to a
synthesised name (e.g. <src>_subcube_x..y..z..fits,
<src>_m{order}.fits).
Lifecycle endpoints
Endpoint |
Behaviour |
|---|---|
|
Enumerate every regular file in the workspace, sorted newest-first. Returns |
|
Stream the artefact back as |
|
Remove the artefact. Idempotent: returns |
Every lifecycle endpoint resolves the filename via
resolve_workspace_path(), which refuses any path containing a separator
or .. segment.
Client surface (DataHubWidget)
The Data Hub’s Workspace Exports panel (buildWorkspaceExportsPanel())
lists the workspace with per-row buttons:
Open — emits
openWorkspaceExportRequested(absolutePath)which MainWindow forwards todoOpenDataPath()(the same flow used for every other dataset open).Download… —
QFileDialog::getSaveFileName+BackendClient::downloadExportrunning inQtConcurrent.Delete — confirm +
BackendClient::deleteExport+ refresh.
The panel auto-refreshes on every backend health tick
(DataHubWidget::refreshStatus()); cube viewers also emit
workspaceExportsChanged() after a successful export so the panel
updates within frames of the operation instead of on the next 4s tick.
Channel Maps (ChannelMapsWindow)¶
Displays an N × M grid of 2-D channel slices with a shared colour scale,
rendered with QCustomPlot (QCPColorMap) instead of VTK to avoid the
macOS OpenGL context limit (~16 simultaneous contexts — a 64-cell grid
would crash). Each cell is a lightweight raster widget.
Component |
File |
Role |
|---|---|---|
Config dialog |
|
Start/End/Stride/Columns/LUT picker (gradient preview icons via |
Mosaic window |
|
Non-modal |
LUT mapping |
|
Samples a |
Data flow: one BackendClient::requestSubvolume(did, 0, W-1, 0, H-1, z0, z1)
call (extended with range_min/max, spectral_axis_type/unit, bunit)
fetches the full z-slab. Client-side slicing extracts each stride-selected
plane and populates the QCPColorMap::data() cells. The shared data
range (rangeMin..rangeMax from the subvolume response) is applied to
every colour map so the LUT is directly comparable across panels.
Double-click → enlarged view: opens a QDialog with a single
full-size QCPColorMap + QCPColorScale (colour bar). The gradient is
passed by value from the mosaic (not copied from the source cell, because
QCPColorMap::setColorScale() resets the gradient to the scale’s
default — the gradient must be set after the colour-scale link).
Drag + zoom are enabled for detail inspection.
PNG export: iterates over cells, resize() + replot() each to
the export geometry (400 × 340 @ 2× DPR) before toPixmap() so
QCPTextElement title labels (“CH 28”) lay out at the export width
rather than the on-screen widget size. Composited into a single QImage
with a file-name + range title bar.
Image viewer (vtkWindowImage) — recent additions¶
Several features were ported from (or inspired by) the cube viewer:
Feature |
Implementation |
Notes |
|---|---|---|
Beam indicator |
|
Hidden when beam keywords are absent. |
Region ExclusiveOptional |
|
Same fix as the cube viewer — prevents multiple shapes checked simultaneously. |
WCS SegmentedToggle |
Coordinate format ( |
Toggle writes into the original |
Linear / Log scale |
Old |
Same pattern as cutting-plane Show Contours in the cube sidebar. |
Contour overlay |
|
|
FITS export → Workspace |
|
Simpler than the cube’s crop/moment flow — just a file copy with collision-safe basename. |
Measurement tools |
|
ExclusiveOptional QActionGroup; deactivates probe/region on enter. |
Pixel histogram |
|
Samples with stride for large images; separate floating window ( |
Annotations |
Text ( |
|
Blink / Compare |
|
|
Contour Phase 2 (cube → image) |
|
No WCS reprojection — assumes shared pixel grid (same dataset). |
Stokes analysis |
|
Master layer NaN colour set to transparent for radio mosaic convention. |
Debiased P |
|
Label includes the σ value used for traceability. |
Spectral index |
|
Both layers must share extent — explicit check, no resampling. |
Faraday RM |
|
No PA unwrapping — valid only in the low-RM regime. Documented limitation. |
Polarization vector overlay |
|
Pixel-grid orientation assumes N-up E-left (standard FITS); rotated WCS would need a correction matrix. |
Radio region stats |
|
Median-MAD σ replaces std-dev for fields with bright sources. |
Catalogue 3D viewer (vtkWindowCatalogue3D)¶
Renders a remote CSV catalogue as a 3-D point cloud in Cartesian (RA/Dec/distance) space.
Key capabilities:
Coordinate frame: FK5 J2000 (default) or Galactic (l, b) via
applyFrameToEntries(); useswcscon()from libwcsDistance resolution (per entry, priority order):
entry.distanceMpcoverride (set by cosmology selector)catalogue
distance/dist/dMpcfieldcosmological integration from redshift (
z,REDSHIFT,Zspec, …)hardcoded fallback 300 Mpc
Cosmology model selector: Planck18 (local integration), Planck15/13/WMAP9 (async batch via
/v1/cosmology/distance/batch)Geometry modes: Ellipsoid, Sphere, Point, Cross (vtkGlyph3D)
Size modes: Fixed, Major axis, LLS, Flux
Interaction: hover highlight (yellow wireframe sphere), click-select (red wireframe), sidebar info panel, table view dock
Morphology LUT: deterministic colour per morphology class; unknown classes cycle the palette
Pagination: 50 000 row pages; “Load more” button
Loading placeholder¶
All viewers show a centred vtkTextActor message while data is being fetched:
Viewer |
Text |
Disappears when |
|---|---|---|
|
“Loading…” (3D) / “Loading…” (2D slice) |
|
|
“Loading image…” |
|
|
“Loading…” |
end of |
|
“Loading…” |
end of |
Style: font 16 pt, colour (0.72, 0.84, 0.91) (#B8D6E8), centered, NormalizedViewport (0.5, 0.5), no bold, no shadow.
VBT viewer (vtkWindowVbt)¶
Renders a remote VBT (VisIVO Binary Table) dataset as a 3-D point cloud.
Key capabilities:
Render modes: Plain (vtkPolyDataMapper) and Gaussian splat (vtkPointGaussianMapper)
Color mapping: any scalar field; configurable colour map and range
Sidebar: Display (render mode, colour, range), Filters, Metadata pages
Pagination: 50 000 row pages; “Load more” appends column vectors
HiPS viewer (HiPSWindow)¶
Interactive all-sky survey browser.
Key capabilities:
Fetches AllSky mosaic via
requestHiPSAllsky(); individual tiles viarequestHiPSTile()requestHiPSTilesForView()requests the backend to compute which tiles cover the current viewportAstronomical name resolution via
resolveTarget()→POST /v1/resolve/targetCatalogue overlay via
requestHiPSCatalogueOverlay()→POST /v1/hips/catalogue_overlayHiPSViewportWidgetowns the tile compositing and paint logic
Tool dialogs (uniform modality)¶
All “tool” dialogs launched from menus or the Tools sidebar are now opened
non-modal (QDialog::show()), so the user can keep interacting with the
viewers, panels, and other tools while a long compute is running. The list
covers LinewidthDialog, BaselineDialog, StackDialog, SourceFindDialog,
LUTCustomizerDialog (2D + 3D), the FITS header viewer, and the moment
description popup. Long-running operations show their own scoped progress
indicator inside the dialog (a QProgressDialog window-modal to the dialog
itself, not to the application) so the “busy” state is still clear.
StackDialog is built around an explicit DatasetEntry list:
The caller (
MainWindoworvtkWindowCube) callsBackendClient::listSessionDatasets(sessionId)in a worker, gets the full set of cubes open in the backend session, and passes them with thecurrentDatasetIdof the calling window.The dialog pre-checks the current cube, hides non-cube entries, and disables (with tooltip) cubes whose
(width × height × depth)does not match the reference shape — they cannot be stacked together.Item label = basename of the FITS file; the true
dataset_idlives inQt::UserRole.
Heavy-task throttle (backend pool sharing)¶
The single ProcessPoolExecutor in the backend (size VISIVO_WORKERS,
default 4) is shared by every endpoint. A global asyncio.Semaphore
(_HEAVY_SEM) in backend/app/dependencies.py caps how many long-running
“heavy” invocations can hold a pool slot at once, leaving the rest free for
interactive requests:
Helpers:
_run,_run_with_limit(interactive);_run_heavy,_run_heavy_with_limit(heavy, semaphore-gated).Default heavy capacity:
max(1, VISIVO_WORKERS - 1).Classification: moment / isosurface / PV / spectral (linewidth, baseline, stack) → heavy; preview / slice / subvolume / noise / image → interactive.
Net effect: a slice scroll, ROI subvolume, or probe issued while a moment /
linewidth / stack is running is processed by the free worker instead of
queueing behind the long job. The chunk dispatch inside the linewidth
orchestrator also goes through _run_heavy, so its parallelism is bounded
by the semaphore rather than by the pool size.
Tunables: VISIVO_HEAVY_SLOTS, VISIVO_LINEWIDTH_CHUNKS,
VISIVO_LINEWIDTH_SNR (see docs/async-patterns.md).
Unit Tests (tests/)¶
QTest-based headless suite; no VTK, no Qt::Widgets, no live backend.
File |
Tests |
What it covers |
|---|---|---|
|
14 |
|
|
24 |
|
Build:
cmake -B build -DBUILD_TESTING=ON
cmake --build build --target VisIVOTests
ctest --test-dir build -V
Static library visivo_test_support (BackendClient + DiagnosticsManager) is shared across test executables without pulling in Qt::Widgets or VTK.
Stable Boundaries Today¶
Boundary |
Status |
|---|---|
|
stable; all backend process management goes through here |
|
stable; Backend → Auth → Ready sequence before main window |
|
local, explicit request/result |
|
local, explicit request/result |
|
backend-authoritative facade |
|
stable; all backend I/O goes through here |
|
stable singleton; all structured logging goes through here |
Async worker pattern |
stable; each viewer owns its watchers explicitly |
Pagination state |
stable; uniform across catalogue and VBT viewers |
Loading placeholder |
stable; all four viewers use the same |
Layer alignment pipeline |
stable; all layer sources (manual, remote, VLKB) route through |
Non-Goals / Deferred¶
No dataset upload/staging for desktop-local files (moment computation requires backend-visible dataset)
No remote rendering
No general async/job framework (each use case defines its own watcher)
No cancellation or progress reporting for in-flight requests
No generic service interfaces (DI, plugin system)
No large
vtkWindowCubedecomposition
Architecture Decisions¶
1. All backend I/O through a single synchronous client called off-thread¶
BackendClient is synchronous and safe to call from QtConcurrent::run. No async Qt Network code, no callback spaghetti. Each worker creates its own BackendClient instance.
2. UI thread only reads results, never calls the backend directly¶
The UI thread is responsible only for applying result structs to VTK pipelines and widget state. It never blocks on network I/O.
3. Pagination is stateful per-window, reset on filter change¶
m_currentOffset / m_totalCount are window-local. applyFilter() always resets them. The “Load more” button is the only way to advance the offset.
4. Cosmology distances are transparent to rendering code¶
Catalogue3DEntry::distanceMpc (0 = auto) is the override seam. entryDistanceMpc() applies the priority chain. Rendering code calls only entryDistanceMpc() and is unaware of which source was used.
5. Build modularisation follows dependency direction¶
visivo_shared_core → no Qt::Widgets, no VTK.
visivo_shared_vtk → no Qt::Widgets.
GUI executable → can use all.
Test suite → links only visivo_test_support (core subset).
Performance Tuning¶
Backend environment variables¶
Variable |
Default |
Effect |
|---|---|---|
|
|
Number of threads for M0/M6/M10 moment chunking. |
|
|
Cube byte size above which the Dask out-of-core path is selected (when Dask is installed). |
|
|
Number of ProcessPoolExecutor workers for CPU-bound FITS operations. |
|
|
Max concurrent heavy tasks (moment / isosurface / pv / spectral). Leaves |
|
|
Row-chunks the linewidth orchestrator fans out per request. Already gated by |
|
|
Per-pixel SNR cutoff for skipping background pixels before the Gaussian fit ( |
|
|
LRU capacity of the in-process product cache (shared by moment, isosurface, pv, linewidth results). |
Client build flags (CMake)¶
Flag |
Default |
Effect |
|---|---|---|
|
|
Enable the optional VR (OpenXR) cube viewer offload. Requires a VTK built with |