When Multilingual is enabled and VHM is enabled too, serving any folderish's folder_contents under a language folder as a top level domain name is currently broken (if en.site.com/archives is VHMed to PloneSite/en/archives then @@qsOptions URL is /archives/@@qsOptions instead of being /@@qsOptions as it should be.

The fundamental problem is that the function get_top_site_from_url(...) is failing to retrieve the actual top site because it can't find a site in the traversal structure, so it picks the last folderish as the site, which is obvs broken because /@@qsOptions only works on site roots, not on folderish content.

This code makes it so that any site found in the structure is returned as the top site from the URL, but if no top site is found in that traversal, the actual site from getSite() is returned instead, which is the correct behavior based on how the function is named, and actually fixes the /@@qsOptions conundrum.

Reproducer

Odd intersection of circumstances to reproduce bug:

1. Create classic Plone site.
2. Enable Multilingual with en-US or some other language.
3. Verify visiting /Plone redirects to Plone/en-US or your language folder of choice.
4. VHM your site so that /Plone/en-US is accessible as / (a standard, supported Multilingual rewrite configuration typically used by Ploners for language-code subdomain hosting). Plone/en-US/VirtualHostRoot is the relevant VHM rewrite.
5. Visit the site. Verify that site is accessible as /.
6. Create folder xxx on that location.
7. Visit folder xxx on that location.
8. Click on Folder contents in the editor toolbar.
9. Verify that the folder contents screen for xxx loads, but the actual listing remains empty.
10. Verify that there is an HTTP 404 for @@qsOptions in the network panel of your browser inspector.

Rationale

This code makes the method sometimes return a content object in some cases, rather than an actual site root.

Why?

All callsites (see below) need a "root object" from the perspective of the browser to compute a root and generate various URLs for their own purposes, because all their purposes are XHR browser stuff (fill lists of objects and search results, chiefly). This method I'm fixing determines the "root" used for that tree which is used in related items, image insertion, and folder contents browsing, and if that "root" is incorrect / does not correspond to what the browser sees as "root", nothing works.

Needless to say, this method should never be used to traverse anything server-side.

Quality control

Callsites

My omelette is big. There are not that many uses, and all its uses currently broken are fixed by this patch. I count two uses:

1. plone.app.content.browser.contents:ContentsBaseAction / get_options / ContextInfo.call
2. plone.app.widgets:get_relateditems_options

Unit and regression testing

All current unit tests pass now. I have added a test case to cover the "root URL object is not actually a site" case, which was never tested before. In addition to that, I have tested this with a local build (most up to date code) and here is what I can verify:

• Related items works now!
• Image insertion works as well.
• Browsing folders in folder contents works too.

So pretty much everything that was broken before, is no longer broken thanks to this modest change. And that is the complete manual regression test of all users of this code.

My previous fix broke several tests. This appears to be the result of trying to traverse iteratively through the structure of a site (which my code does and the previous code does not do in the same way), and encountering Unauthorized exceptions during traversal (which is a legitimate thing to get, as /a/b/c may be accessible to user XXX, but /a/b may not necessarily be).

The fix is to use unrestricted traversal (bypassing security), and then to return the selected object by using restricted traversal in that case.

I am confident this should not result in a security issue — however I ask the true expert Plonistas to verify this confidence is warranted. Thanks.

And sorry for the breakage. We need CI tests in plone.base.

This plugin should not be used at all, but at least not by default. It does everything with inline styles. The recommended way to do table styling is with the formats, similar to how we did it since Plone 4.

This lets ulocalized_time: accept a string argument to long_format, for example: ${a} ${d} hello guys ${b} ${Y}. It is a tiny change, really just one extra if-statement. Fixes https://github.com/plone/Products.CMFPlone/issues/3549.

Also supports dollar signs in registry override for date formats. Then it uses the correct language in multilingual sites. Fixes https://github.com/plone/Products.CMFPlone/issues/3550.

Adds lots of tests for current behavior and for the new fixes/features.

In some cases (when an object does not yet exist, for example) this method being called must skip these nonexistent objects from the recursive evaluation being done to discover the appropriate site to return.

We should catch it when we miss a dependency, or notice when a dependency pulls in too many others. We currently miss the plone.schema dependency, needed for the Email field. Reported in issue #9, which needs a solution first.

We currently miss the plone.schema dependency, needed for the Email field. If we add it, it pulls in plone.app.z3cform for IPloneFormLayer, and this package pulls in lots of plone.app packages.

@jensens Do you think we should move IPloneFormLayer to plone.base? Wait: then plone.schema would need to import it from plone.base, causing a circular dependency between the two. Bad idea. Move it to plone.z3cform then?