in reply to Domino Twisty Expansion using Lightweight Proxy

I've never seen someone write this before but that may not indicate much. Mostly I don't think this is the sort of thing a Domino developer would care about and so there isn't likely to be lots of code around to extract the contents of documents via the HTTP service. Actually, now that I think on this further there's a better way.

Use the normal Domino APIs to extract the data without even bothering with the web service. You're going a very twisty way when you could either go get a RichText object, translate it to text (with optional formatting) and index that. Another idea is to just use the normal FullText index. It sounds like you are trying to avoid using Domino's pre-existing indexing service. Why? You'd have to be crazy to give it up - it already works really darn well.

As for your current problem - I'd transform the HTML to XML with XML::LibXML and just use some XPath to fetch the expansion URLs. The in-place editing might be somewhat ugly so now I wonder if providing a more expansive URL command like ExpandAll might not be a better idea anyway.

  • Comment on Re: Domino Twisty Expansion using Lightweight Proxy

Replies are listed 'Best First'.
Re: Re: Domino Twisty Expansion using Lightweight Proxy
by inman (Curate) on Feb 13, 2004 at 16:58 UTC
    The search engine that I use to index the content has a connector for Lotus Notes that sucks the content out using the Notes API. This works very well for Domino servers that are hosted on our internal network.

    The Twisty problem still causes an issue when a document from a user search is viewed. At this time, the doc is generated as HTML by Domino and streamed through the search engine so that keyword highlights can be added etc. The users get annoyed when the search engine returns results that they can't immediately see because they are hidden in a twisty.

    Another problem is that there are a large number of Domino servers hosted elsewhere (external hosting provider etc.) that I cannot connect to using the Notes API. These servers must be indexed via HTTP and the representation of the 'ExpandSection' argument plays havoc with the indexing. The search spider views each version of the URL as a new document.

    The 'ExpandAll' argument is something that Domino web developers have been asking about for years. Lotus just couldn't do it in R5 and it can't be done using LotusScript. The Perl app that I am developing is the most recent in a long line of potential solutions.

[reply]
      I was extrapolating ExpandAll from the view URLs - I've not tried to do it against a document before (but then our web standards generally don't put things like collapsable sections onto documents). Oh well. Good luck with the HTML fetching!
[reply]

In Section Seekers of Perl Wisdom