[deepamehta-devel] Question about "org.neo4j.graphdb.NotInTransactionException" when creating TopicType

mre at deepamehta.de mre at deepamehta.de
Mon Oct 20 18:14:49 CEST 2014


Hi dear Martin, Hi dear dm-developers!

To bring us onto the next level I allowed myself to extract three
questions out of our latest dialogue (quoted below) regarding the further
development of the "Freifunk API Directory"- (or "Freifunk API Query"-)
Client.

1 An illustration (see image attached) of our current topic type model for
thinking about "(re-)creating this topic type model somehow generically" on
import

2 An (unprioritized but) list of types missing towards mapping the
freifunk api model to deepamehta4 1:1

3 What I think the "wilderness" of the data structure in the ff-summarized
directory is.


And here are my answers:

1: DM4 has builtin the concept of an "evolitionary data model" so I dont
see changing APIs as a (big) challenge. Genereting the topic type model
on-the-fly though on the other hand (also for me) would be quite a big
effort. Furthermore, see my answer to 3, on what I see as the main
challenge processing FFN API Data.

The (slightly annotated) image attached shows what our declarative
migrations currently create as the topic type model. This graphic may help
you and me to wrap our head around the necessary effort and discuss
"re-creation of this topic type model" on every import/update. Annotation:
I manually added the cardinalities in between the types to the screenshot.
The rest of the screenshot shows our application model revealed "as is"
with the help of the the standard dm4-webclient.

2: Please have a look at the list of un-processed data fields (extracted
of the ffn-directory-summary-json) at
https://github.com/freifunk/query.api.freifunk.net/issues/5 and give/or
gather some user feedback regarding useful next steps.

3: Regarding "wilderness": Currently we must (3.1) check if there is a key
in the json before accessing the json-values (to not fail hard during
processing of the json document) and if so, (3.2) just then access the
json-value with given key. Now, at least when processing the summarized
directory the main challenge I see: We currently do not know in advance if
the values for a given key and that ffn-community are _many_ (json-array)
or _one_ (e.g. json-string), making accessing/processing the entries in
this directory especially cumbersome (adding control flow level 3.3) on
each value to be processed) (see [1]).

My request would therefore be: If, for example, there is just one
"firmware version" in use but in other communities there are maybe more in
use, for reasons of consistency and thus to allow easier processing, always
store values under such key in a list/json-array. When implementing this
query-api, I find this issue more practically relevant than the (maybe more
challenging) issue discussed under nr. 1.

Additionally: If this (3) "wilderness/cardinality of values"-issue is
rooted in the Freifunk API endpoints (and just passed on by the
summarized-directory we currently process), we should think of implementing
just one (cumbersome parser) solution for that in the dm4-freifunk-api
module directly (instead of having the summarized directory in between the
query-api and the endpoints). This (additionally) would give the
dm4-freifunk-api module a bit more freedom regarding, when and what is to
be updated exactly.

Cheers & Looking forward to your feedback on what's to do next [2].

[1]
https://github.com/freifunk/query.api.freifunk.net/commit/1c2861c907f03ab4ff55919e3ae264e88cd66f32
[2] https://github.com/freifunk/query.api.freifunk.net/issues/5

On Sat, 13 Sep 2014 03:15:12 +0200, Martin Tippmann
<martin.tippmann at gmail.com> wrote:
> Hi everyone!
> 
> 2014-09-12 20:20 GMT+02:00 Malte Reissig <mre at deepamehta.de>:
>> And because of your question regarding the use of "Jackson": Since
>> DeepaMehta 4 already ships with Jettison, I would ditch another JSON
>> lib, yes. Jettison is pretty straightforward to use and we anyway have
>> to be parsing the whole API directory very carefully (manually) cause
of
>> its wilderness. Lets get in touch somewhen soon for solving the open
>> questions regarding how to complete the application model in
>> dm4-freifunk-api for your queries.
>>
> 
> I'll write you a mail on the topic. I've made some progress on my
original
> idea to just parse the JSON schema for creating the topic types and I
and
> Andi think that's probably the best way. The schema and the data is
prone
> to rather a lot of changes and maintaining a fork in Java of the schema
is
> something I'd rather would avoid. I'm quite late on the problem but I
think
> it's still a feasible way to go.
> 
> The basic idea is to parse the schema and create the topic types (likely
> for multiple versions) and then parse the current API data and import
> things. The big advantage of this approach would be - if it works - that
> getting the data into DeepaMehta is no problem in the future. All moving
> parts are the API schema and the community data.
> 
> I hope I have some basic implementation ready by Sunday and then I'll
see
> how good this will work.


-------------- n�chster Teil --------------
Ein Dateianhang mit Bin�rdaten wurde abgetrennt...
Dateiname   : dm4-freifunk-api-plugin-model-01.png
Dateityp    : image/png
Dateigr��e  : 265938 bytes
Beschreibung: nicht verf�gbar
URL         : <http://lists.deepamehta.de/pipermail/devel-lists.deepamehta.de/attachments/20141020/78c56f30/attachment.png>


More information about the devel mailing list