Paperwork 2.0: testing phase

Announcements
, ,
#1

Hello,

Good news everyone, what-will-be-Paperwork-2.0 (Paperwork 1.99.x currently) has entered testing phase.

Testing phase is about testing Paperwork but also about translating and documenting it. Previous testing phases lasted about a month. Paperwork 2.0 is a full rewrite, so this testing phase will probably last longer.

Paperwork 1.99 is now in the branch ‘testing’ (changes will be merged periodically in the branch ‘develop’ too). You can get it using Flatpak:

flatpak --user uninstall work.openpaper.Paperwork  # if it is already installed
flatpak --user install https://builder.openpaper.work/paperwork_testing.flatpakref

What’s New ?

Since it is a full rewrite, it’s hard to list all the changes exhaustively. The main changes are:

  • Entirely modular design:
    • Pretty much any features can be disabled (flatpak run work.openpaper.Paperwork plugins list)
    • Code is cleaner, therefore contributions will probably be easier to make
    • You can expect faster and easier improvements later on
  • Better performances:
  • Command-line interface that kicks ass
  • Semi-automatic bug report system that kicks ass (I’m going to need it …)
  • Editable PDFs
  • Smaller exported PDFs
  • Automatic scan border guessing that … doesn’t really kick ass (yet)
  • Various GUI changes

So What Now ?

There are still many things that need to be done before releasing Paperwork 2.0. Of course contributions are welcome.

All the changes listed below must go directly to the Git branch ‘testing’. I will periodically merge the changes from the branch ‘testing’ to the branch ‘develop’.

Specific to GNU/Linux distributions

I need your help to complete:

Testing and stabilization

Feel free to test it and report any bug you find on the bug tracker or using the integrated bug report system.

Documentations

I’m going to update/complete:

  • the installation instructions (the various README.markdown)
  • the user documentation (LaTeX --> PDF --> integrated in Paperwork) (if someone else want to do it, the position is open)
  • and then the developer documentation

Translations

Translations must now be done in Weblate. You can freely register on it.

After updating the documentation, I’m going to translate Paperwork in French (if someone else want to do it, this position is also open). It’s also the perfect time to translate Paperwork in other languages.

There are 5 components to translate:

Do not translate paperwork-help-documents yet as documentations have yet to be completed.

If you need contexts or screenshots for some translations, don’t hesitate to ask me, either in the Paperwork category of this forum or by email. Beware that if you just add comments on Weblate, I am not notified.

If you want to translate Paperwork in languages that are not yet enabled in Weblate, just tell me and I’ll enable them.

Thanks in advance for any help you can provide,

Best regards,

#2

To the person who submitted suggestions for French translations:
If you want your name in the credits, please submit translations directly instead of submitting suggestions. You cannot submit translations anonymously so you will have to create a login on https://translations.openpaper.work/ .

#3

Hello Jerome,

I installed it now on my productive system, so I am going to use it in my daily work. All bugs from the early development phases seem to be fixed. The speed is amazing. I had one crash in the beginning and will file the bugreport. Thanks for your great work.
I will support the translation to german.

Kind regards,
Andreas

1 Like
#4

@AFoster

I got 2 bug reports. I’m not sure which one is yours (or if both are). By default, bug reports from Paperwork are entirely anonymous. For privacy reasons I won’t publish the links to the bug reports here. Next time, if you want to discuss bug reports submitted automatically, can you please either post the link to the bug report or put your name in the bug report description ?

Anyway, the first bug reports contain logs showing that the Whoosh Index or the SQL database are locked. The only thing that comes to mind that could cause that, is that there are 2 instances of Paperwork running side by side. I’ll have to implement a system of lock file to prevent that and warn the user.

The second bug report is about sudden crashes and contains no useful traces. I assume this is because the Sane backend or Libinsane segfaulted. In that case, it makes the Python interpreter crash immediately too, which prevent any logging. I plan on working around that problem later on.
Anyway I’ve configured the whole thing to dump C Python stacktraces on stderr when a crash happens. If it is your bug report, you can try to run Paperwork from a terminal and reproduce the crash. You should then get the stacktraces.

#5

Hi Jerome, yes, both came from me.

I reproduced the sudden crash by importing a file and then scanning another page to append to this document.

The output on the shell is the following:

[INFO  ] [paperwork_backend.docscan.libinsane] [LibInsane] ../subprojects/libinsane/src/normalizers/resolution.c:L231(opt_desc_filter): No change to do on option 'resolution'
Fatal Python error: Segmentation fault

Thread 0x00007facd5c07700 (most recent call first):
  File "/app/lib/python3.7/site-packages/openpaperwork_gtk-1.99.1-py3.7.egg/openpaperwork_gtk/widgets/progress/__init__.py", line 89 in _thread
  File "/usr/lib/python3.7/threading.py", line 870 in run
  File "/usr/lib/python3.7/threading.py", line 926 in _bootstrap_inner
  File "/usr/lib/python3.7/threading.py", line 890 in _bootstrap

Thread 0x00007facd6686700 (most recent call first):
  File "/usr/lib/python3.7/threading.py", line 296 in wait
  File "/usr/lib/python3.7/queue.py", line 170 in get
  File "/app/lib/python3.7/site-packages/openpaperwork_core-1.99.1-py3.7.egg/openpaperwork_core/thread/pool.py", line 24 in run
  File "/usr/lib/python3.7/threading.py", line 926 in _bootstrap_inner
  File "/usr/lib/python3.7/threading.py", line 890 in _bootstrap

Thread 0x00007facd6e87700 (most recent call first):
  File "/usr/lib/python3.7/threading.py", line 296 in wait
  File "/usr/lib/python3.7/queue.py", line 170 in get
  File "/app/lib/python3.7/site-packages/openpaperwork_core-1.99.1-py3.7.egg/openpaperwork_core/thread/pool.py", line 24 in run
  File "/usr/lib/python3.7/threading.py", line 926 in _bootstrap_inner
  File "/usr/lib/python3.7/threading.py", line 890 in _bootstrap

Thread 0x00007facdca3d700 (most recent call first):
  File "/app/lib/python3.7/site-packages/pyocr-0.7.3.dev11+g625b2a2.d20200706-py3.7.egg/pyocr/tesseract.py", line 303 in run_tesseract
  File "/app/lib/python3.7/site-packages/pyocr-0.7.3.dev11+g625b2a2.d20200706-py3.7.egg/pyocr/tesseract.py", line 367 in image_to_string
  File "/app/lib/python3.7/site-packages/paperwork_backend-1.99.1-py3.7.egg/paperwork_backend/guesswork/ocr/pyocr.py", line 176 in ocr_page_by_url
  File "/app/lib/python3.7/site-packages/paperwork_backend-1.99.1-py3.7.egg/paperwork_backend/guesswork/ocr/pyocr.py", line 62 in _run_ocr_on_page
  File "/app/lib/python3.7/site-packages/paperwork_backend-1.99.1-py3.7.egg/paperwork_backend/guesswork/ocr/pyocr.py", line 74 in _run_ocr_on_modified_pages
  File "/app/lib/python3.7/site-packages/paperwork_backend-1.99.1-py3.7.egg/paperwork_backend/guesswork/ocr/pyocr.py", line 78 in add_obj
  File "/app/lib/python3.7/site-packages/paperwork_backend-1.99.1-py3.7.egg/paperwork_backend/sync.py", line 248 in _transaction_simple
  File "/app/lib/python3.7/site-packages/openpaperwork_core-1.99.1-py3.7.egg/openpaperwork_core/promise.py", line 233 in _threaded_do
  File "/app/lib/python3.7/site-packages/openpaperwork_core-1.99.1-py3.7.egg/openpaperwork_core/thread/__init__.py", line 27 in do
  File "/app/lib/python3.7/site-packages/openpaperwork_core-1.99.1-py3.7.egg/openpaperwork_core/thread/pool.py", line 27 in run
  File "/usr/lib/python3.7/threading.py", line 926 in _bootstrap_inner
  File "/usr/lib/python3.7/threading.py", line 890 in _bootstrap

Thread 0x00007facdd23e700 (most recent call first):
  File "/app/lib/python3.7/site-packages/paperwork_backend-1.99.1-py3.7.egg/paperwork_backend/docscan/libinsane.py", line 169 in scan
  File "/app/lib/python3.7/site-packages/openpaperwork_core-1.99.1-py3.7.egg/openpaperwork_core/promise.py", line 233 in _threaded_do
  File "/app/lib/python3.7/site-packages/openpaperwork_core-1.99.1-py3.7.egg/openpaperwork_core/thread/__init__.py", line 27 in do
  File "/app/lib/python3.7/site-packages/openpaperwork_core-1.99.1-py3.7.egg/openpaperwork_core/thread/pool.py", line 27 in run
  File "/usr/lib/python3.7/threading.py", line 926 in _bootstrap_inner
  File "/usr/lib/python3.7/threading.py", line 890 in _bootstrap

Thread 0x00007facf142d740 (most recent call first):
  File "/usr/lib/python3.7/site-packages/gi/overrides/GLib.py", line 497 in run
  File "/app/lib/python3.7/site-packages/openpaperwork_gtk-1.99.1-py3.7.egg/openpaperwork_gtk/mainloop/glib.py", line 63 in mainloop
  File "/app/lib/python3.7/site-packages/openpaperwork_core-1.99.1-py3.7.egg/openpaperwork_core/__init__.py", line 459 in call_one
  File "/app/lib/python3.7/site-packages/paperwork-1.99.1-py3.7.egg/paperwork_gtk/main.py", line 188 in main_main
  File "/app/lib/python3.7/site-packages/paperwork-1.99.1-py3.7.egg/paperwork_gtk/main.py", line 214 in main
  File "/app/bin/paperwork-gtk", line 11 in <module>

I hope that helps.

During my work today, I had a freeze of the window. That happened after some queries, I cannot reproduce it. There is no relevant info in the logfiles.

One question related to the translation: How can I use the translation as far as it is available right now, to see the strings live in the GUI?

Kind regards,

Andreas

#6

Yep, here it is. Most likely a crash caused by the Sane backend of your scanner (or maybe Libinsane :/).

I think a dedicated process would help for this kind of issue (the underlying assumption is that Sane backends are usually tested in minimalist programs, not so much in Python3+GTK+multithreaded applications). I’ll also see if I can add a way to get the C stacktraces easily.

#7

Weblate pushes the translations in Git from time to time (I’m still unclear regarding the when and why). Then CI/CD pipelines automatically rebuild and publish Paperwork in Flatpak. You can simply update Paperwork and you should get them (flatpak --user update). Just make sure to use the branch testing and not develop.

#8

@AForster : Regarding your latest bug report, I’ve opened the ticket #925. At the moment, everything looks OK so I have no idea why it doesn’t work in German but works in French.

#9

Good news, the help documents are now complete. You can now translate paperwork-help-documents.

If you see any mistake, typo, poor phrasing, etc, please don’t hesitate to report them.