[extension-v1.19.0] - 2024-12-21: Reworked popup UI, better replay integration

Changed (1)

- Popup UI:

- Reorganized the whole layout by assigning tags to all elements and allowing switching between those tags as if they were tabs.

The original idea was to unroll in steps a-la uBlock Origin, but this is superior.

- Improved some help strings.

Added

- Core + Popup UI + Shortcuts:

- Added Replay from the archiving server configuration option.

It’s a tristate of: disallow, enable if Submit dumps via 'HTTP' option is enabled and the server supports it, enable even if Submit dumps via 'HTTP' option is disabled.

- Added Include in global replays per-tab options.

- Added popup UI button and keyboard shortcut both of which re-navigate all tabs for which Include in global replays is set to their replays.

- Added popup UI button, keyboard shortcut, and context menu item all of which re-navigate a currently active tab to its replay.

- Added Force 'Work offline' in replayed tabs configuration option which does the same thing the similar options for file: and data: URL does, but for tabs that point to replay URLs. Enabled by default.

- Added 🎄 Winter Days mode seasonal theme.

- Added Escape notification messages configuration option to help support more notification daemons. Disabled by default.

Changed (2)

- The Help page:

- Merged “Handling of failures” section into “Archival”.

- Reworded some awkward places.

- Core + manifest.json:

- Improved server checking logic and error messages.

- Improved keyboard shortcut descriptions.

- Improved documentation.

Fixed

- Core:

- Snapshot buttons and keyboard shortcuts will no longer take DOM snapshots of replay pages, unless Capture snapshots of all URLs option is set.

- On Chromium, fixed Hoardy-Web trying to collect and archive replay pages.

This release integrates the extension with tool-v0.20.0, which can now do both archival and replay over HTTP, see below.

Changed

- Core:

- From now on, all requests to all URLs under Server URL will be ignored, allowing you work with tool-v0.20.0-replayed pages without fiddling with any settings.

- From now on, the extension will respect archiving server’s settings and features given by its /hoardy-web/server-info endpoint, if such a thing exists.

- The default value of Server URL does not specify /pwebarc/dump endpoint anymore, as this is now configurable server-side.

For old configs, you can keep the old value, the archiving server handling code will silently elide that path away.

- From now on, before the first archival, the extension will check that a working archiving server is available at the given Server URL and generate errors describing what exactly appears to be broken when not.

- Popup UI:

- From now on, if you set Server URL setting to an empty string, it will be reset to the default value.

- Improved CPU usage when switching tabs really quickly.

See CHANGELOG.md (also there).

Changed

- Popup UI:

- Reverted most of the block reordering bit of popup UI rework of extension-v1.17.0.

The “Globally” block is near the top again.

- Edited the “Persistence” block a bit more.

Mainly, to stop graying out always-useful stat lines, even when the associated features are disabled, to prevent possible confusion there.

- Renamed some options and stat lines, mostly to make their names shorter to make popup UI on Fenix more readable.

- Toolbar button:

- Edited its title format to be much shorter, especially on Fenix.

- Reverted the ordering of parts there to how it was before extension-v1.17.0.

The (much shorter now) “globally” part is at the front again because otherwise the badge being at the front there too without an explanation of its format is kind of confusing.

- Core + All internal pages:

- Improved message handling infrastructure.

- Used it to improve initialization functions of all internal pages, improving efficiency and making the resulting UI much less flaky.

- The Help page:

- Documented what webNavigation permission is used for, improved the rest a bit.

- *:

- Renamed build.sh firefox target to firefox-mv2, for consistency.

Fixed

- UI:

- Fixed flaky rendering of Help and Changelog pages on Fenix.

They render properly now the very first time you load them, no reloads needed.

- Fixed duplication of history entries when navigating internal links.

- Fixed source links sometimes failing to being highlighted when pressing the browser’s “Back” button.

- Fixed some small CSS nitpicks.

- Popup UI + Documentation:

- Realigned some help strings with reality.

- Fixed some more mostly inconsequential things.

Halloween special: major UI and state display improvements, fine-grained `Work offline` mode, add-on reloading with its state preserved, new options, etc. A huge number of changes. See CHANGELOG.md (also there) for more info.

Fixed

- On Chromium, fixed request tracking being frequently broken since extension-v1.15.0.

- Fixed reqres without responses but with networking errors having “Responded at” field set in the logs.

Changed

- Renamed pWebArc -> Hoardy-Web.
- Renamed all ::pWebArc:: error codes into a more consistent naming scheme.
- Improved documentation.

Fixed

- Fixed some typos.

Changed

- Improved notifications.
- Improved documentation.

Added

- pWebArc is now officially supported on Fenix (Firefox for Android). It is quite usable there now, so go forth and test it.

- Chromium version now has a update_url set in the manifest.json, so if you use chromium-web-store or some such, it can be updated semi-automatically now, see the extension’s README.md for more info.

- Implemented User Interface and Accessibility > Verbose option.

From now on, by default, pWebArc will have its most common but annoying notifications mention they can be disabled and explain how.

This is mostly for Fenix users, where these things are not obvious, but it could also be useful for new users elsewhere.

- Implemented User Interface and Accessibility > Spawn internal pages in new tabs option which controls if internal pages should be spawned in new tabs or reuse the current window.

It can not be disabled on desktop browsers at the moment, but is disabled by default on mobile browsers.

- Implemented a bunch of new notifications about automatic fixes applied to config.

I.e., it will now not just fix your config for you, but also complain if you try to set an invalid combinations of options.

- Implemented Generate desktop notifications about ... > UI hints option to allow you to disable the above notifications.

Changed

- pWebArc will CBOR-dump all reqres fields completely raw from now on.

wrrarms learned to handle this properly quite a while ago.

This simplifies the parsing of the results and makes the implementation adhere to the stated technical philosophy more closely.

The dumps will grow in size a tiny bit, but this is negligible, since they are compressed by default by all the archival methods now.

Moreover:

- A lot of UI improvements, mainly for Fenix.
- From now on, per-tab Stash 'in_limbo' reqres option is being inherited by children tabs like the rest of similar options do.
- Renamed build.sh chromium target to chromium-mv2 in preparation for eventual chromium-mv3 support.
- Advanced minimum browser versions to Firefox v102, and Fenix v113.
- Improved performance.
- Improved documentation and installation instructions.

Fixed

- Fixed wrong “In limbo” counts after the extension gets reloaded.
- Fixed race conditions in browserAction updates.
- Worked around browserAction title updates being flaky on Fenix. I.e. you can stare at the Extensions > pWebArc line in the browser’s UI now while the browser fetches some stuff and it will be properly interactively updated.
- On Firefox, fixed the id of the extension leaking into origin_url field of the very first dump of each session when Workarounds for Firefox bugs > Restart the very first request is enabled (which is the default).
- Fixed some typos.

Added

- pWebArc now runs under Fennix aka Firefox-for-Android-based browsers, including at least Fennec and Mull.

Thought, “Export via saveAs” archival method is broken there, because of a bug in Firefox. Other methods do work, though.

- The above change also added a settings page (aka options_ui).

At the moment, the settings page is simply an unrolled by default version of popup UI, with per-tabs settings removed.

This is need because on mobile browsers the main screen of the browser is not a tab and there’s no toolbar, so there’s no popup UI button there, and so the extension UI becomes really confusing without a separate settings page.

Changed

- Split in_flight stat into a sum of two numbers.

This makes things less confusing on Chromium, the “Help” page explains it in more detail.

- Added toolbar button’s badge as a prefix to its title, changed its format a bit.

This is needed because Fennix-based browsers do not display the badge at all, so this change helps immensely there. Meanwhile, on desktop browsers this does not hurt.

- Improved styling and dark mode contrast of the popup UI.

- Improved documentation.

In particular, among other things, added a lot of new anchors to the “Help” page, most internal links referencing some fact discussed in another section now point directly to the relevant paragraph instead of pointing to its section header.

Fixed

On Firefox:

- Fixed capture of responses produced by service/shared workers.

Also, added a new error code for when it (very rarely) fails because of a race condition inherent in webRequest API and documented all of it on the “Help” page.

- Fixed HTTP protocol version detection, requests fetched via HTTP/3 will now be marked as such.

- Added yet another webRequest API error to a list of those that mark reqres response data as incomplete.

On Chromium:

- Fixed more edge cases where reqres could get stuck in in_flight state indefinitely.

Generally:

- Fixed navigation with browser’s “Back” and “Forward” buttons to work properly on the “Help” page.

- Fixed a bug where force-stopping all in-flight reqres in a single tab could also drop some of the others.

Fixed

- Fixed a lot of places where the documentation was misaligned with current reality.

Changed

- Improved documentation, especially the “Help” page.
- Tiny improvement in popup UI HTML layout.
- Changed config.history default value.

Added

- Implemented reqres persistence across restarts.

pWebArc can now save and reload collected but not archived reqres (including those in_limbo) by stashing them into browser’s local storage. This is now enabled by default, but it can be disabled globally, or per-tab.

- As a consequence, pWebArc now tracks browsing sessions and shows when a reqres belongs to an older session on its “Internal State” page.

- Implemented two new archiving methods, now making the total of three. I.e. pWebArc can now archive collected reqres by

- generating fake-Downloads containing either separate dumps (one dump of an HTTP request+response per file) or bundles of them (many dumps in a single file, for convenience, to be later imported via wrrarms import bundle),

- archiving separate dumps to your own private archiving server (the old one, the previous default, inherited on extension update),

- archiving separate dumps to your browser’s local storage (the new default on a new clean install).

- As a consequence, pWebArc now has a new “Saved in Local Storage” page for displaying the latter.

- Implemented display and filtering for queued and failed reqres on the “Internal State” page.

- Implemented tracking of per-state size totals for reqres in most states after finished.

- As a consequence, popup UI will now display those newly tracked sizes.

- Introduced the errored reqres state.

With stashing to local storage enabled, pWebArc will not try its best not to loose any captured data even when its archiving code fails (bugs out) with an unexpected exception. If it bugs out in the capture code, then all bets are off, unfortunately.

Changed

- pWebArc will now track if an error is recoverable and will not retry actions with unrecoverable errors automatically by default.

- pWebArc now follows the following state diagram:

(See CHANGELOG.md in the source repository.)

- Renamed all “Profile” settings into “Bucket”, as this makes more sense.

- Improved popup UI layout.

- Changed toolbar icon’s badge format a bit.

- Improved debugging options.

- A huge internal refactoring to solve constant sub-task scheduling issues once and for all.

- Improved documentation.

Removed

- Removed config.logDiscarded option as it is no longer needed (pWebArc has proper log filtering now).

Fixed

- Various small bugfixes.

- pWebArc will no longer automatically reload on updates, waiting for the browser to restart or for you to reload it explicitly instead.

This way you won’t lose any data on extension updates.

Proper automatic reloads on updates will be implemented later, after pWebArc gets full persistence.

- Popup UI:

- Reverted the split between “Globally” and “This session”.

Implementing that split properly will make future things much harder, so, simple is best.

- “Queued” stat moved to a separate line again.

It also shows the sum total of sizes of all dumps now.

- Added “Scheduled … actions” stat line, showing the names of actions that are scheduled.

It is hidden by default, because watching it closely while pWebArc is very busy can probably cause seizures in some people.

- Improved documentation.

- Bugfixes.

- Implemented DOM snapshots, their popup UI, keyboard shortcuts and documentation.

- Ctrl+Alt+S runs snapshotTab by default now.

- Changed some default keyboard shortcuts:

- Ctrl+Alt+A and Ctrl+Alt+C run collectAllInLimbo and collectAllTabInLimbo respectively now;
- Alt+Shift+D and Alt+Shift+Wrun discardAllInLimbo and discardAllTabInLimbo now.

- Popup UI:

- Improved layout.
- Destructive actions will start asking for confirmations now.

- Added a bunch of new toolbar icons for various tab states.

In particular, problematic state as well as mixed-capture states (e.g., disabled in this tab, but enabled and with limbo mode in children tabs) now have their own special icons.

- All SVG icons were edited to not reference any fonts, since those are not guaranteed to be available on a user’s system.

- Improved behaviour of new tabs created by clicking buttons on the “Internal State” page.

- Greatly improved documentation.

- Implemented dark mode theme. The extension will switch to it automatically when the browser asks (which it will if you switch your browser’s theme to a dark one).

- Implemented some new optional UI-related accessibility config options with toggles in popup UI:

- Colorblind mode: uses bluish colors instead of greenish where possible (which uses mostly the same colors pWebArc used before color-coding of UI toggles was introduced in v1.9.0, with slight variations for the new color-coding).
- Pure text labels: disables emojis in UI labels, makes screen readers happier.

- Improved Internal State/Log UI:

- Added a bunch of tristate toggles for filtering the logs.
- Added in-log buttons to open a narrowed page for reqres with an associated tab.

- Improved keyboard shortcuts:

- In popup UI, toggles and buttons with bound keyboard shortcuts will now get those shortcuts displayed in their tooltips.
- The “Keyboard shortcuts” section of the “Help” page will now show currently active shortcuts (when viewed via the “Help” button from the extension UI).
- The changes to the code there mean all the shortcuts will be reset to their default keys, but it makes stuff much cleaner internally, so.
- Collecting all reqres from currently active tab’s limbo is bound to Alt+S by default now (similarly to how Ctrl+S saves the page).
- Discarding all reqres from currently active tab’s limbo is bound to Alt+W by default now (similarly to how Ctrl+W closes the tab).
- Unmarking all problematic reqres in the currently active tab is bound to Alt+U by default now.
- Added a few more shortcuts:
- Alt+Shift+U by default unmarks all problematic reqres globally now.
- Alt+Shift+S and Alt+Shift+W by default respectively collect and discard all reqres in limbo globally now.

- Added UI for internal scheduled/delayed actions/functions (e.g., saving of frequently changing stuff to persistent storage, automatic actions when a tab closes, etc):

- If some functions are still waiting to be run, the badge will have “~” or “.” in it and change its color, depending on the importance of the stuff that is waiting to be run.
- Popup UI has a new stat line showing the number of such delayed actions and buttons to run or cancel them immediately.

- Added config options and popup UI toggles for picking and marking as problematic reqres with various HTTP status codes.

- “Mark reqres as problematic when they finish … with reqres errors” config option became “… with reqres errors and get dropped”, i.e. it is now disjoint with “… with reqres errors and get picked”.

- Implemented new config options and popup UI toggles for browser-specific workarounds.

- Improved desktop notifications, added some more of them, added config options and popup UI toggles for them.

... and more. See `CHANGELOG.md` in the source repository for more info.

- Bugfixes. A whole ton of bugfixes.

- pWebArc now follows a new state diagram.

Terminology-wise, most notably, picked and dropped now mean what collected and discarded meant before.

See the “Help” page for more info.

- A lot of changes to make pWebArc consistently use the above terminology — both in the source and in the documentation — were performed for this release.

- New features:

- Implemented “negative limbo mode”.

It does the same thing as limbo mode does, but for reqres that were dropped instead of picked. (Which is why there is an arrow from dropped to in_limbo on the diagram above.)

- Implemented optional automatic actions when a tab gets closed.

E.g., you can ask pWebArc to automatically unmark that tab’s problematic reqres and/or collect and archive everything belonging to that tab from limbo.

- Implemented a bunch of new desktop notifications.

- Added a bunch of new configuration options.

This includes a bunch of them for controlling desktop notifications.

- Added a bunch of new keyboard shortcuts.

Also, keyboard shortcuts now work properly in narrowed “Internal State” pages.

- Implemented stat persistence between restarts.

You can brag about your archiving prowess to your friends by sharing popup UI screenshots now.

- Added the “Changelog” page, which can be viewed by clicking the version number in the extension’s popup.

- Improved visuals:

- Extension’s toolbar button icon, badge, and title are much more informative and consistent in their behaviour now.

- The version number button in the popup (which opens the “Changelog”) will now get highlighted on major updates.

- Similarly, the “Help” button will now get highlighted when that page gets updated.

- The popup, the “Help” page, the “Internal State” aka the “Log” page all had their UI improved greatly.

- All the toggles in the popup are now color-coded with their expected values, so if something looks red(-dish), you might want to check the help string in question just in case.

- Improved documentation.

- Tiny bugfix.

- Implemented "problematic" reqres flag, its tracking, UI, and documentation.

This flag gets set for "no\_response" and "incomplete" reqres by default but, unlike "Archive reqres with" settings, it does not influence archival, instead pWebArc d
isplays "error" as its icon and its badge gets "!" at the end.

This is needed because, normally, browsers provide no indication when some parts of the page failed to load properly --- they expect you to actually look at the page
with your eyes to notice something looking broken instead --- which is not a proper way to do this when you want to be sure that the whole page with all its resources was
archived.

- Implemented currently active tab's limbo mode indication via the icon.

- Added more shortcuts, changed defaults for others:

- Added `toggle-tabconfig-limbo`, `toggle-tabconfig-children-limbo`, and `show-tab-state` shortcuts,

- Changed the default shortcut for `collect-all-tab-inlimbo` from `Alt+A` to `Alt+Shift+A` for uniformity.

- Renamed reqres states:

- `noresponse` -\> `no_response`,
- `incomplete-fc` -\> `incomplete_fc`.

- Added a separate state for reqres that are completed from cache: `complete_fc`.

- Improved UI, the internal state/log page is much nicer now, but the popup UI in its default state might have become a bit too long...

- Improved performance when using limbo mode.

- Improved documentation.

- Bugfixes.

(Actually, this releases about half of the new changes in my local branches, so expect a new release soonish.)

- Implement "in limbo" reqres processing stage and toggles.

"Limbo" is an optional pre-archival-queue stage for finished reqres that are
ready to be archived but, unlike non-limbo reqres, are not to be archived
automatically.

Useful in cases when you need to actually look at a page before deciding if
you want to archive it.

E.g., you enable limbo mode, reload the page, notice there were no updates
to the interesting parts of the page, and so you discard all of the reqres
newly generated by that tab via appropriate button in the add-on popup, or
via the new keyboard shortcut.

- The "Log" page became the "Internal State" page, now shows in-flight and
in-limbo reqres, and allows narrowing to data belonging to a tab.

- Improved UI.
- Improved performance.

- Replaced icons with a cuter set.

- Added shortcuts toggling tab-related config settings.
- Improved UI.
- Improved documentation.
- Bugfixes.

- Improved document_url and origin_url handling.
- Implemented context menu actions.
- Improved UI.
- Improved efficiency when dumping to CBOR.
- Improved documentation.

Bug fixes, UI improvements, experimental Chromium support.

Better handling of "304 Not Modified" responses, UI and help improvements, bugfixes.

Fixed some small bugs, better popup/settings UI, the "Help" page is much more helpful and the "Log" page updates automatically now.