cd

Hi all,

just wondering if anyone has toyed with the underlying H2 database of PB-files and used HSQL or the like on it.

I'd like to hear from you.

Cheers
chris

--
PB 5.5.2
Java 1.6.0_14 w/ -Dawt.useSystemAAFontSettings=lcd -Dswing.defaultlaf=com.sun.java.swing.plaf.gtk.GTKLookAndFeel -Dawt.toolkit=sun.awt.motif.MToolkit
Crunchbang Linux on an NC10
Quote
cd

ok, might have been too quick. database seems to be password protected. also: it's the old h2-format, so one needs h2-1.2.127.jar .. but all that's not going to do it w/o password.

Any chance one could get this? Well, i don't suppose that accessing the h2-db-files would be against licensing terms of PB, or would it?

chris

--
PB 5.5.2
Java 1.6.0_14 w/ -Dawt.useSystemAAFontSettings=lcd -Dswing.defaultlaf=com.sun.java.swing.plaf.gtk.GTKLookAndFeel -Dawt.toolkit=sun.awt.motif.MToolkit
Crunchbang Linux on an NC10
Quote
b4l4nc3r
I was just connected to this thread by Dys, thanks again.

Sound very interesting, and I don't mind the security from their point of view. I'd be delighted if they at least published a public API to be used on the open and closed brains. That would be amazingly lovely!

Quote
jroeterd

Or a read-only user...

I guess the point of reporting on the DB is to make reports that are very difficult within PB. Sounds very nice.

Jeroen

Quote
cd

I don't think the PB team is interested in this yet. However, time passing they will come to realize the necessity behind such a step. Because let's face it: it can't be that hard to publish an API that does

open, close, addNode(guid, type), findThoughtBy(name,guid,attachement,tag), addAttachmentTo(guid), getLinks(giud), addTag, addTagTo(), getNotes(guid), ... - the basics off the top of my head -

from any location on the planet that has internet access and that works on a running and a non-running instance of PB (esp. the latter case should be easy and is what i am looking into right now if at all possible).

I dislike software that can't attach to anything else. And that in the age of orchestrating one's information and application heaven (or hell). Import/ export stuff is so 90s to quote s.o. close to me.

Or at least: provide us with an xml-structure that is "live" and that i don't have to export every step of the way... An xml-export takes about 5min on one of my brains and don't ask about webbrain... (ok, slow netbook, but still).

The reason i was looking into H2-access (and i haven't stopped) is that i need to be able to take my data along wherever need be. I still am using PB as my main information-storage application and it runs whenever my computer is running. I've also started using it as a calendar reminder for the non-critical stuff and i use it extensively w/ regard to GTD: next action, now, waitingFor etc. Now, when i want the latest data available on the road, i have to export the xml to a server (or create a webbrain that takes forever) and query it through a web interface. How eh, degenerate is that? Instead, i could simply save a complex hsql query (for that PB is no good yet) and run it with a click on a link on an up2date version of the DB as it it on my harddrive/... and visualize that via wap/ html on my cell.

I also want to be able to input stuff into the brain w/o having it in front of me (eg from my cell). To do so, i had to go the great length of creating a mail2pbxml component so that when i come home at night i can import the xml from there into the brain (see one of my threads in this forum). If i forget to import or if stuff changed in the meantime, well, more sorting work and more inconsistencies. Instead i just push the info via hsql onto my "sleeping beauty of a brain.data.db"...

On uservoice i see all those requests and i am sure that most of them are smart ideas. Most of these come down to this: give potential developers the tools at hand to make the software better by e.g. writing plugins or through the use of an API manipulating thoughts. I am not saying to make it open source though that would be fantastic. What i'm asking is simply: give me my data back. I know it's in my PB and i can export it/ etc. BUT: When it takes too much effort to do so, that software has to go. And that would be a shame because PB fills precisely the void between good visual access to your data and sth. like personal datawarehousing. While i think there are better tools out there to focus on issues/ creating overviews on stuff, it is the best i know for accessing data quickly and linking stuff that belongs together easily.

Ask me what i would do with such access? Wait for PB6beta, check out the meta-data functionality if it's included (typed nodes etc) and tap into that potential to cross-link what i'm doing on my pc while PB is running (open apps, open files, ...) as well as what my cell data offers (gps, ...) with the data that's inside PB. I want to be able to add a thought in PB (let's say "milk" underneath "groceries") and my cell to go off reqall-style when I'm near a grocery store that is cross-ref'd in PB as carrying milk. Or i want a picture taken on my cell to appear in PB and that is automatically linked to the city/ person i'm at w/i PB. Or i want PB to automagically open the thought of a person i work with when the person calls me on my phone in realtime (or pinning it for a couple of seconds, or updating a "CONTEXT"-node, yay).

Ultimately, i want fragmentation to stop. Information should flow freely and not be staggering like a drunken man through too small a door...

Oh gosh, another one of those rants where i find no end. Well, i'm passionate about this software and have been from early natrificial days on. So i hope you'll excuse me (and not censure me as done in the past).

I guess it would have been more productive to list all api functions that we would like to see, hehe. Or coming back to H2: getting an answer as to whether pursuing access on the db would be considered illegal by PB. Anyone from PB on that matter?

Anyway, looking forward to you ideas. dinner awaits. Sorry for the long post.
chris

--
PB 5.5.2
Java 1.6.0_14 w/ -Dawt.useSystemAAFontSettings=lcd -Dswing.defaultlaf=com.sun.java.swing.plaf.gtk.GTKLookAndFeel -Dawt.toolkit=sun.awt.motif.MToolkit
Crunchbang Linux on an NC10
Quote
b4l4nc3r
Chris, you can relax now since you got it off your chest, and I too believe it was worth it, I too love to express my self in a good rant from time to time : )
I'm sure the Dalai Lama actually said or quoted someone: "If you don't 'speak passionately, then why speak at all?", it was something in that regard. But that was just to encourage passion, since anyone that has something they want to share, should be able to within the boundaries of healthy reasons, which is relative, but I hope you see where I went with that.

To put things in a clear perspective in which I'll include everyone's current suggestions, therefore I'll invest in this post a little more, just because it might change the way we think with our PB a lot!

Readonly DB user? Or (il?)legal direct DB access? Not recommended, because if they change the DB structure to improve or because of necessity, that will break 3rd party code, thus the users too.

The public API functions might be deprecated if really necessary, but that will give time to adapt without a breakdown!

A full list of functions right now won't be needed, because all we should be asking the PB-team -at first- is an initial basic implementation, which should require much less time from them, then otherwise the case. Which should make the idea a little bit more feasible. Just like the basic implementation of the Saved Extended Views, which doesn't support activation by an Accelerator nor sub-categories, BUT it sure as hell makes my life -and those depending on such views- MUCH MUCH easier. So we should focus our initial efforts in gathering the necessary support to get feedback from the PB-team about their thoughts on the matter.

If they're not interested, then they've a reason. All we need to do is give them a patch for their business model to make this idea more interesting from a closed-code business model point of view. If more common modern "developers" (any code writer) can interact with a PB brain through a public API, then it will make PB much more usable and thus gain a lot more advocates and heavy adopters, like us! Admittedly, since we're individual consumers, we won't be able to individually contribute to the monetary earnings from our own pockets like companies are able to, BUT that's at the same time our strength! There are much more individual consumers than companies, so in share numbers we beat companies hands down, only if enough join the PB movement!

The PB team already supports this view, that's why obviously -and also through their public statements- they started the desktop version of PB in the first place. I read that they do offer a SDK for developers, but the pricing model is meant for companies. If they offer a simplified adapted version to individual consumers for the desktop version of PB or just start with a basic public API, AS AN EXPERIMENT, they will be able to get the necessary feedback to see if it's worth investing in. They should adopt an Experimenters Role on this issue too. They might start with a survey, but I believe this forum is also meant -among other reasons- for such a process, to get feedback from the users.

So I suggest that we start a UV suggestion, and show our support  by voting there, and gathering more supporters.

I believe that it's best to focus on the "Basic PB Public API", and only after that start another UV stating "Extending PB Public API", just to be clear that we won't throw rocks at them for cutting corners on the first implementation, as I know how critical some users can get, aside the justifications, it will just cause unproductive friction.

Feedback?
Quote
b4l4nc3r
I've dimmed down the frictional side of my personality noticeably. I'm stating this just as a clarification for those thinking: "He sure doesn't realize he's talking about himself!"
Quote
cd

b4l4nc3r, i'm with you on starting this small. and i hear you w/ regard to the closed code business model. i've looked at BrainEKP and am glad that sth albeit smaller is available for desktop use. what i am interested in, just as you i suppose, is making the end-user experience better. so i propose the following:

1) Create 2-3 scenarios of a potential future in which PB has a CORE API
  - querying a thought called sth like "East-cost sales" for it's attachments and getting the PDF called yyy as a result of a 2nd query sent via https/ imaps/ ...
  - presentation mode: a series of macros (api calls) is recorded and contains break points/ etc for use in presentations where each macro-series is triggered by a click of the mouse e.g.
  - a location sensing device updates PB and looks for affordances w/i it's thoughts
  - these are just ideas i came up with rapidly, we'll have to think about those
2) Arguing in favor of a Server-type architecture (a daemon so to speak) of PB
  - security wise this might be a BIG step. i still think it's the way to go.
3) creating a Forum in the general section where more people are likely to read it and linking to a uservoice request (that i won't put up there since i don't have an account and don't really see the use of it yet..)

In the meantime, i'd still be interested to know if accessing the H2-db is possible and/or in violation of terms w/ PB license :-) Because i want complex queries and regex matching... in that regard, a read only user would be fine at first.

Cheers
Chris

--
PB 5.5.2
Java 1.6.0_14 w/ -Dawt.useSystemAAFontSettings=lcd -Dswing.defaultlaf=com.sun.java.swing.plaf.gtk.GTKLookAndFeel -Dawt.toolkit=sun.awt.motif.MToolkit
Crunchbang Linux on an NC10
Quote
b4l4nc3r
cd wrote:

Anyway, looking forward to you ideas. dinner awaits. Sorry for the long post.
chris



Btw, in my first comment before writing the longer version, while FF crashed and flushed it out of the memory, I showed my appreciation for the share amount of use cases you've mentioned! I just hope we get any type of feedback from the PB-team so we can attune to it, by first knowing how we should be approaching the situation. It's still too much repeated manual work and time consuming processes right now. It's because despite that it still ROCKS, is the reason why we're sticking to it and hoping it keeps on growing even more toward our wishes!
Quote
cd

Quote: It's still too much repeated manual work and time consuming processes right now. It's because despite that it still ROCKS, is the reason why we're sticking to it and hoping it keeps on growing even more toward our wishes!

i'm sure it will. PB has many advantages vs other apps out there and the learning curve isn't steep. however, other tools are free (vue come's to mind, presentation wise it's a blast) and also rock. in order to keep paying customers happy, a product needs innovation. A forum such as this - and users that are willing to share their experiences as you have correctly noted before, is the gold, the difference that makes a difference and one from which future versions of an application will profit. The advantage of a closed code app could be that focus is more directed. The great disadvantage is that it doesn't make use of all the resources outside the project and that keeping up with user requests might be too much as a side-dish to everyday work, as i find it blurs the edges and restoring focus can be a pain.

I'm glad you liked my use cases. I have more. And i'm willing to share up to a certain point. After that, i need commitment ;-p

--
PB 5.5.2
Java 1.6.0_14 w/ -Dawt.useSystemAAFontSettings=lcd -Dswing.defaultlaf=com.sun.java.swing.plaf.gtk.GTKLookAndFeel -Dawt.toolkit=sun.awt.motif.MToolkit
Crunchbang Linux on an NC10
Quote
Harlan
Hello, anxious PB API developers.

Access to the DB is not going to be possible... PB is far beyond a plain vanilla H2 database. It has been customized and supplemented in many ways and is very complex and subject to change. There are a lot of rules about how PB expects data to be inserted and updated. For example, when thoughts are changed, you need to update all of their inheritied info (icon, label, color) based on their type and its type plus the info that inherits from it. Also, there are many optimization tables that must be updated to enable things like instant search. Plus, the search indexes need to be updated also - and those are not even part of the database. The queries needed to do this are very complex and opening up the db even to experienced experts would surely lead to corruption of the data. Oh and I even forgot - the continuous backup service would be rendered useless if not kept in sync with the info in the db.

Thus, the API is going to be where it's at.

As you'll see in version 6, WebBrain is going to become a more integral part of braining... It includes a built in synchronization capability that makes publishing updates to your brain a very simple and quick process. Only changes will be sent each time you sync , making it quite painless.

Once your brain is online, queries as well as updates will be possible using a web-based API. Using the API will be simple and as easy to use as opening a URL. It will allow you to ask what's connected to a thought, to update a thought, create thoughts and much more.

Changes made to your online brain can then be synced back to your local copy (or copies)...
Regards,
-Harlan
Quote
cd

Harlan, thanks for taking the time to join us.

Quote: As you'll see in version 6, WebBrain is going to become a more integral part of braining... It includes a built in synchronization capability that makes publishing updates to your brain a very simple and quick process. Only changes will be sent each time you sync , making it quite painless.

Once your brain is online, queries as well as updates will be possible using a web-based API. Using the API will be simple and as easy to use as opening a URL. It will allow you to ask what's connected to a thought, to update a thought, create thoughts and much more.

This is GREAT NEWS with one little question mark: Does "Once your brain is online" mean online on any server we wish or only hosted brains? Because the latter makes it more or less impossible to use for private use. Although personal information does not always mean private, a huge portion of PIM-data is private and still needs to be accessed. Trust then is an issue... (not necessarily trust in the company, but also trust in networks, webapps, ...). 

Apart from that: Can't wait to see PB6 (beta? when? :-p) in action.

Re DB: an API will make this unnecessary. So THANKS!

Cheers
chris

--
PB 5.5.2
Java 1.6.0_14 w/ -Dawt.useSystemAAFontSettings=lcd -Dswing.defaultlaf=com.sun.java.swing.plaf.gtk.GTKLookAndFeel -Dawt.toolkit=sun.awt.motif.MToolkit
Crunchbang Linux on an NC10
Quote
b4l4nc3r
  Harlan wrote: Hello, anxious PB API developers.

Access to the DB is not going to be possible... PB is far beyond a plain vanilla H2 database. It has been customized and supplemented in many ways and is very complex and subject to change. There are a lot of rules about how PB expects data to be inserted and updated. For example, when thoughts are changed, you need to update all of their inheritied info (icon, label, color) based on their type and its type plus the info that inherits from it. Also, there are many optimization tables that must be updated to enable things like instant search. Plus, the search indexes need to be updated also - and those are not even part of the database. The queries needed to do this are very complex and opening up the db even to experienced experts would surely lead to corruption of the data. Oh and I even forgot - the continuous backup service would be rendered useless if not kept in sync with the info in the db.

Thus, the API is going to be where it's at.

First I want to thank you for taking time out of the PB development to update the users!
I understand the complexity for "direct DB access", but what about "public desktop API"? I mean if the PB code does the work, wouldn't it be possible for it to offer publicly callable methods/functions to be easily used by the user/developer? Meaning methods which would encapsulate the complexity into the individual classes?

  Harlan wrote:
As you'll see in version 6, WebBrain is going to become a more integral part of braining... It includes a built in synchronization capability that makes publishing updates to your brain a very simple and quick process. Only changes will be sent each time you sync , making it quite painless.

Once your brain is online, queries as well as updates will be possible using a web-based API. Using the API will be simple and as easy to use as opening a URL. It will allow you to ask what's connected to a thought, to update a thought, create thoughts and much more.

Changes made to your online brain can then be synced back to your local copy (or copies)...

I see, I understand it's much smarter to invest in the web implementation of the public API, since it includes a much broader set of use cases.
But for updating the local brain by API calls, one will have to go around it and update the web brain and sync back, instead of immediate bulk work on the active brain at the speed of the local computer only most limited by the HDD speed, especially when a lot of data is involved. Of course it depends on the speed of the sync, as well as how much time it needs to analyze what is changed, if it needs to do that in any case. I guess will find out in the beta testing.

Being able to sync a brain partly -through filters or the like- would really be needed for people like me that use a massive central one, and only want to share partly. On a relevant note, in any case, I still would need the requested feature for individual exports and the other use cases: "Add Visible Thoughts to the Selection-pane" http://forums.thebrain.com/post?id=4692470&pid=39992810#post39992810
It would be great if it made it into version 6, if it's not too much work, or it doesn't clash with your point of views. If it does, then please give some feedback on that regard over there.
Quote
b4l4nc3r
cd wrote: Because the latter makes it more or less impossible to use for private use. Although personal information does not always mean private, a huge portion of PIM-data is private and still needs to be accessed. Trust then is an issue... (not necessarily trust in the company, but also trust in networks, webapps, ...).

I -and I believe many other users too- are with Chris on this one how will the public web API be practically usable when one uses a central brain for both "private" and "sharable" data?
Quote
cd

The simple answer is: it is not. I cannot imagine putting a part of the data i need on a hosted solution when what i truly need is access to data when i need it where i need it and in a secure way. Carrying around a netbook isn't a solution in most cases and i'm sure plenty of users have sensible data in their brains.

This solution, if our deductions are at all correct, simply does not compute for most of us. If that is the path chosen, then TB will need to address privacy issues quickly: make brains encrypted/ add detailed syncing mechanisms/ ensure the security of hosted brain data etc... (zero knowledge policy if you ask me). One solution could be to have a mobile client that syncs _private_ thoughts with the desktop version and thus complements the web-api-sync-round dance but i don't think it would be wise.

Anyway, let's wait and hear and better yet experience how this is going to turn out. I'm sure glad i was wrong with parts of my assumptions on PB policy. A Web Api is a first step, even if its reality lies restrained in the confines of a hosted system. Any maybe, just maybe, there's a servlet waiting for us out there...

chris

--
PB 5.5.2
Java 1.6.0_14 w/ -Dawt.useSystemAAFontSettings=lcd -Dswing.defaultlaf=com.sun.java.swing.plaf.gtk.GTKLookAndFeel -Dawt.toolkit=sun.awt.motif.MToolkit
Crunchbang Linux on an NC10
Quote

Newsletter Signup  Newsletter Signup        Visit TheBrain Blog   Visit TheBrain Blog       Follow us on Twitter   Follow Us       Like Us on Facebook   Like Us         Circle Us on Google+  Circle Us         Watch Us on Youtube  Watch Us       

TheBrain Mind Map & Mindmapping Software     Download TheBrain Mind Mapping Software