[orientdb] Cool History solution: How can I save a new (history) ODocument from within a Hook

[Thomas]

Hi,

Using orientdb-graphed-1.0.1-SNAPSHOT I'm trying to create a history
solution using Hooks.

From inside the hooks (ONLY onRecordBeforeUpdate and onRecordBeforeDelete)

I want to do the
following which work except for the final save operation :-(

1. Copy a document that is about to be saved or deleted. The copy will be
used to create a history document.
2. Get all dirty fields from the document that is about to be updated and
then
3. Copy the original values from the ODocument (if any exist) and finally
4. save the document to a History cluster.

@Override
public boolean onRecordBeforeUpdate(ORecord<?> iRecord) {
if( iRecord instanceof ODocument ){
ODocument doc = (ODocument) iRecord;
if(validateClusterId(doc)) {
ODocument copy = doc.copy(); // Contains dirty fields that
we need to update with the old values before we can save the document to
History.
String[] dirtyFields = doc.getDirtyFields();
for(int i=0;i<dirtyFields.length;i++){
if(doc.getOriginalValue(dirtyFields[i]) == null)
System.out.println("Field [ " + dirtyFields[i] + " ] did not exist in
previous version");
else {
System.out.println("Original Value of dirty field [
" + dirtyFields[i] +
" ] will be copied to History version.
Original value to be copied to the History version is [ " +
doc.getOriginalValue(dirtyFields[i]) + " ]
New value is [ " + doc.field(dirtyFields[i]) + " ]");
copy.field(dirtyFields[i],
doc.getOriginalValue(dirtyFields[i]));
}
}
copy.save("History");
}
}
return true;
}


The problem is the final save operation which give the following exception:

com.orientechnologies.orient.core.exception.OConcurrentModificationException:

Cannot update record #8:0 in storage 'orientdb' because the version is not
the latest.
Probably you are updating an old record or it has been modified by another
user (db=v1 your=v0)

I have also tried to use doc.getDatabase() and then create a new Vertex or
Edge, but this is not possible.


Any suggestions on how to save from inside the Hook?


Thanks

[Thomas]

Hi again,

I have just seen the following page about the exception described above.
Looking into that.

http://code.google.com/p/orient/wiki/TroubleshootingJava#OConcurrentModificationException:_Cannot_update_record_#X:Y_in_s

If I find a solution I will get back.

See ya

[Thomas]

This post might be inappropriate. Click to display it.

[Luca Garulli]

Hi,
the problem is that the copied record has a RID, so an update is always
executed instead of a create. Just reset the RID:

copy.getIdentity().reset();

or manage things in different way by always creating a new ODocument and
copy the fields then.

Post by Thomas
-Dcache.level1.enabled=false
and
OGlobalConfiguration.CACHE_LEVEL1_ENABLED.setValue(false);
-Ddb.mvcc=false
Disabling the MVCC system worked (I don't get the OConcurrentModificationException any
more) but when i search the graph for any ODocuments in the History cluster it's empty.
database.query(new OSQLSynchQuery[ODocument]("select from cluster:History"))
Any ideas?
Thanks

[Thomas]

Hi Luca,

Once again I Thank You.

You have been a great help :-)

[Luca Garulli]

Hi Thomas,
I think your use case it's quite common for most of developers. Probably it
could be useful providing this feature bundled in OrientDB. As hook? As a
new annotation?

Once it works we could start with a WiKi page with this use case under a
new section "Use cases". WDYT?

Post by Thomas
Hi Luca,
Once again I Thank You.
You have been a great help :-)

[Thomas]

For anyone interested, the modification to the previously posted hook
callback is shown below.

Notice the *copy.getIdentity().reset();* proposed by Luca
and the deletion of fields that did not exist in the version of the
document before update.


@Override
public boolean onRecordBeforeUpdate(ORecord<?> iRecord) {
if( iRecord instanceof ODocument ){
ODocument doc = (ODocument) iRecord;
if(validateClusterId(doc)) {
ODocument copy = doc.copy(); // Contains dirty fields that
we need to update with the old values before we can save the document to
History.
copy.getIdentity().reset();
String[] dirtyFields = doc.getDirtyFields();
for(int i=0;i<dirtyFields.length;i++){
if(doc.getOriginalValue(dirtyFields[i]) == null) {
System.out.println("Field [ " + dirtyFields[i] + "
] did not exist in previous version. Removing this field from the History
version.");
copy.removeField(dirtyFields[i]);
}
else {
System.out.println("Original Value of dirty field [
" + dirtyFields[i] +
" ] will be copied to History version.
Original value to be copied to the History version is [ " +
doc.getOriginalValue(dirtyFields[i]) + " ]
New value is [ " + doc.field(dirtyFields[i]) + " ]");
copy.field(dirtyFields[i],
doc.getOriginalValue(dirtyFields[i]));
}
}
copy.save("History");
}
}
return true;
}

Hope it may help someone out there :-)

[Luca Garulli]

Probably it could be useful adding a new field called "date" storing a new
Date() to know when the version has been created.

Post by Thomas
For anyone interested, the modification to the previously posted hook
callback is shown below.
Notice the *copy.getIdentity().reset();* proposed by Luca
and the deletion of fields that did not exist in the version of the
document before update.
@Override
public boolean onRecordBeforeUpdate(ORecord<?> iRecord) {
if( iRecord instanceof ODocument ){
ODocument doc = (ODocument) iRecord;
if(validateClusterId(doc)) {
ODocument copy = doc.copy(); // Contains dirty fields that
we need to update with the old values before we can save the document to
History.
copy.getIdentity().reset();
String[] dirtyFields = doc.getDirtyFields();
for(int i=0;i<dirtyFields.length;i++){
if(doc.getOriginalValue(dirtyFields[i]) == null) {
System.out.println("Field [ " + dirtyFields[i] + "
] did not exist in previous version. Removing this field from the History
version.");
copy.removeField(dirtyFields[i]);
}
else {
System.out.println("Original Value of dirty field
[ " + dirtyFields[i] +
" ] will be copied to History version.
Original value to be copied to the History version is [ " +
doc.getOriginalValue(dirtyFields[i]) + " ]
New value is [ " + doc.field(dirtyFields[i]) + " ]");
copy.field(dirtyFields[i],
doc.getOriginalValue(dirtyFields[i]));
}
}
copy.save("History");
}
}
return true;
}
Hope it may help someone out there :-)

[Mich Geni]

Hi,

And you may also opt to create a link (a property containing a link to say
class 'Issue') from the latest (HEAD) node to immediate previous version of
the node and so on, to create a linked-list (containing history nodes in a
chain). So that when you want to query for a version of a record say #8:230
by date, it can easily be done.

example: #8:230 -> #100:221 -> #100:220 -> #100:219 and so on
backwards date-wise. (where #8:230 is a latest record of type 'Issue' and
others are previous versions of the same record in a separate cluster
id=100 for history)


Regards,

Mich

Post by Luca Garulli
Probably it could be useful adding a new field called "date" storing a new
Date() to know when the version has been created.

Post by Thomas
For anyone interested, the modification to the previously posted hook
callback is shown below.
Notice the *copy.getIdentity().reset();* proposed by Luca
and the deletion of fields that did not exist in the version of the
document before update.
@Override
public boolean onRecordBeforeUpdate(ORecord<?> iRecord) {
if( iRecord instanceof ODocument ){
ODocument doc = (ODocument) iRecord;
if(validateClusterId(doc)) {
ODocument copy = doc.copy(); // Contains dirty fields
that we need to update with the old values before we can save the document
to History.
copy.getIdentity().reset();
String[] dirtyFields = doc.getDirtyFields();
for(int i=0;i<dirtyFields.length;i++){
if(doc.getOriginalValue(dirtyFields[i]) == null) {
System.out.println("Field [ " + dirtyFields[i] +
" ] did not exist in previous version. Removing this field from the History
version.");
copy.removeField(dirtyFields[i]);
}
else {
System.out.println("Original Value of dirty field
[ " + dirtyFields[i] +
" ] will be copied to History version.
Original value to be copied to the History version is [ " +
doc.getOriginalValue(dirtyFields[i]) + "
] New value is [ " + doc.field(dirtyFields[i]) + " ]");
copy.field(dirtyFields[i],
doc.getOriginalValue(dirtyFields[i]));
}
}
copy.save("History");
}
}
return true;
}
Hope it may help someone out there :-)

[Luca Garulli]

Yes,
probably this could be the basis for the Time Machine feature:
http://code.google.com/p/orient/issues/detail?id=180

Post by Mich Geni
Hi,
And you may also opt to create a link (a property containing a link to say
class 'Issue') from the latest (HEAD) node to immediate previous version of
the node and so on, to create a linked-list (containing history nodes in a
chain). So that when you want to query for a version of a record say #8:230
by date, it can easily be done.
example: #8:230 -> #100:221 -> #100:220 -> #100:219 and so on
backwards date-wise. (where #8:230 is a latest record of type 'Issue' and
others are previous versions of the same record in a separate cluster
id=100 for history)
Regards,
Mich

Post by Luca Garulli
Probably it could be useful adding a new field called "date" storing a
new Date() to know when the version has been created.

Post by Thomas
For anyone interested, the modification to the previously posted hook
callback is shown below.
Notice the *copy.getIdentity().reset();* proposed by Luca
and the deletion of fields that did not exist in the version of the
document before update.
@Override
public boolean onRecordBeforeUpdate(ORecord<?**> iRecord) {
if( iRecord instanceof ODocument ){
ODocument doc = (ODocument) iRecord;
if(validateClusterId(doc)) {
ODocument copy = doc.copy(); // Contains dirty fields
that we need to update with the old values before we can save the document
to History.
copy.getIdentity().reset();
String[] dirtyFields = doc.getDirtyFields();
for(int i=0;i<dirtyFields.length;i++){
if(doc.getOriginalValue(**dirtyFields[i]) == null) {
System.out.println("Field [ " + dirtyFields[i] +
" ] did not exist in previous version. Removing this field from the History
version.");
copy.removeField(dirtyFields[**i]);
}
else {
System.out.println("Original Value of dirty
field [ " + dirtyFields[i] +
** " ] will be copied to History
version. Original value to be copied to the History version is [ " +
** doc.getOriginalValue(**dirtyFields[i])
+ " ] New value is [ " + doc.field(dirtyFields[i]) + " ]");
copy.field(dirtyFields[i], doc.getOriginalValue(
**dirtyFields[i]));
}
}
copy.save("History");
}
}
return true;
}
Hope it may help someone out there :-)

[Mich Geni]

And using Graph API, we can have more interesting and direct linking:

(considering the above mentioned example in this thread earlier)

create an edge with a Date() property between:

#8:230 ------Date ------> #100:221
#8:230 ------Date ------> #100:220
#8:230 ------Date ------> #100:219
..
#8:230 ------Date ------> #100:110

so traversal is fast because of direct connections between each history
node and latest (HEAD) node #8:230

(or optionally the linked list way, described earlier. I think we can give
user an option to choose which?)

Points to ponder:

- The linked-list of connecting history nodes makes looks-up linear and
slow, but do not create burden on 'out' and 'in' properties of history
enabled vertices and edges
- The direct (links/edges) between history nodes (vertices) looks
apparently fast, but what is the performance of having thousands of links
in 'out' and 'in' properties for such (history enabled) vertices and edges?
(SIDENOTE: *For efficient Graph API and traversal, we should have no
performance penalty for having too many (e.g. millions) edges between (any
two or more) vertices.*)


(I continued the discussion in this thread here, as we can give a link to
this thread in issue: http://code.google.com/p/orient/issues/detail?id=180)


Regards,

Mich

Post by Luca Garulli
Yes,
http://code.google.com/p/orient/issues/detail?id=180

Post by Mich Geni
Hi,
And you may also opt to create a link (a property containing a link to
say class 'Issue') from the latest (HEAD) node to immediate previous
version of the node and so on, to create a linked-list (containing history
nodes in a chain). So that when you want to query for a version of a record
say #8:230 by date, it can easily be done.
example: #8:230 -> #100:221 -> #100:220 -> #100:219 and so on
backwards date-wise. (where #8:230 is a latest record of type 'Issue' and
others are previous versions of the same record in a separate cluster
id=100 for history)
Regards,
Mich

Post by Luca Garulli
Probably it could be useful adding a new field called "date" storing a
new Date() to know when the version has been created.

Post by Thomas
For anyone interested, the modification to the previously posted hook
callback is shown below.
Notice the *copy.getIdentity().reset();* proposed by Luca
and the deletion of fields that did not exist in the version of the
document before update.
@Override
public boolean onRecordBeforeUpdate(ORecord<?**> iRecord) {
if( iRecord instanceof ODocument ){
ODocument doc = (ODocument) iRecord;
if(validateClusterId(doc)) {
ODocument copy = doc.copy(); // Contains dirty fields
that we need to update with the old values before we can save the document
to History.
copy.getIdentity().reset();
String[] dirtyFields = doc.getDirtyFields();
for(int i=0;i<dirtyFields.length;i++){
if(doc.getOriginalValue(**dirtyFields[i]) == null) {
System.out.println("Field [ " + dirtyFields[i]
+ " ] did not exist in previous version. Removing this field from the
History version.");
copy.removeField(dirtyFields[**i]);
}
else {
System.out.println("Original Value of dirty
field [ " + dirtyFields[i] +
** " ] will be copied to History
version. Original value to be copied to the History version is [ " +
** doc.getOriginalValue(**dirtyFields[i])
+ " ] New value is [ " + doc.field(dirtyFields[i]) + " ]");
copy.field(dirtyFields[i], doc.getOriginalValue(
**dirtyFields[i]));
}
}
copy.save("History");
}
}
return true;
}
Hope it may help someone out there :-)

[Thomas]

Hi Luca and Mich,


The original idea was that all Vertices and Edges have a mandatory
timestamp field of cause,

and the idea was also to treat Vertices and Edges the same (with a
mandatory timestamp ONLY).


I think that it is bad programming practice to start carrying around
history information in

the Vertices and Edges (Sorry Mich - no offense). I'm not to crazy about
the timestamp solution

myself, but I thing that this is the best solution.


How do we update the timestamp for a modified Vertex or Edge? Originally I
thought

that this would be done before an update (before the hook is called), but
we could of cause

automate it and have the timestamp updated inside the hook. If we do it
outside the hook

however, the posted code will take care of copying the old timestamp to the
history version

of the Vertex or Edge, but I agree, why do it manually if we can automate
it inside the hook.

There is however one important reason for updating the timestamp from
outside the hook, and

that is if we need to keep working with the document which may cause
inconsistency between

the version in memory and the version in storage (Maybe - I don't know if
this could become a

problem).


In the timestamp solution we need however to take into account when a
Vertex or Edge

is deleted. We need a non-mandatory field e.g. deletetimestamp if we do not
want to loose

history information about when a document was deleted. The original
timestamp only tells

us when a document was last updated (this is saved to History) but since we
are deleting

the document we only have the time for last modification, not the
deletetimestamp. Thus

we need to add a deletetimestamp to the last History document before
deletion.


If you think it through, the simplicity of the proposed timestamp/History
cluster solution

is very beautiful. We can always search the history cluster and find out
exactly how a Vertex

or Edge was connected and what the field values were at any given time.


Unless you are frequently working specifically with History I do not think
that we should

store other information than a simple mandatory timestamp. Note that no
"active" Vertex or

Edge (i.e. not in the History cluster) will contain the deletetimestamp.
Only a mandatory

timestamp indicating when it was created.


By the way Luca, I think it would be really cool to create a wiki page.
Can you give me an idea where to start :-)


Note: If you use the History cluster solution along with clustering in
general

you need to have cluster pairs such as
InventoryCluster/InventoryClusterHistory,

CusotmerCluster/CusotmerClusterHistory etc. if you do not want to loose
cluster

information in the history space.


Best regards,


Thomas

[Luca Garulli]

Hi Thomas,
I've just created the Use Case page:
http://code.google.com/p/orient/wiki/UseCases and
http://code.google.com/p/orient/wiki/UseCaseHistorical for your use case.

Please give me your google code account to be enlisted between the
contributors to modify the WiKi.

Post by Thomas
Hi Luca and Mich,
The original idea was that all Vertices and Edges have a mandatory
timestamp field of cause,
and the idea was also to treat Vertices and Edges the same (with a
mandatory timestamp ONLY).
I think that it is bad programming practice to start carrying around
history information in
the Vertices and Edges (Sorry Mich - no offense). I'm not to crazy about
the timestamp solution
myself, but I thing that this is the best solution.
How do we update the timestamp for a modified Vertex or Edge? Originally
I thought
that this would be done before an update (before the hook is called), but
we could of cause
automate it and have the timestamp updated inside the hook. If we do it
outside the hook
however, the posted code will take care of copying the old timestamp to
the history version
of the Vertex or Edge, but I agree, why do it manually if we can automate
it inside the hook.
There is however one important reason for updating the timestamp from
outside the hook, and
that is if we need to keep working with the document which may cause
inconsistency between
the version in memory and the version in storage (Maybe - I don't know if
this could become a
problem).
In the timestamp solution we need however to take into account when a
Vertex or Edge
is deleted. We need a non-mandatory field e.g. deletetimestamp if we do
not want to loose
history information about when a document was deleted. The original
timestamp only tells
us when a document was last updated (this is saved to History) but since
we are deleting
the document we only have the time for last modification, not the
deletetimestamp. Thus
we need to add a deletetimestamp to the last History document before
deletion.
If you think it through, the simplicity of the proposed
timestamp/History cluster solution
is very beautiful. We can always search the history cluster and find out
exactly how a Vertex
or Edge was connected and what the field values were at any given time.
Unless you are frequently working specifically with History I do not
think that we should
store other information than a simple mandatory timestamp. Note that no
"active" Vertex or
Edge (i.e. not in the History cluster) will contain the deletetimestamp.
Only a mandatory
timestamp indicating when it was created.
By the way Luca, I think it would be really cool to create a wiki page.
Can you give me an idea where to start :-)
Note: If you use the History cluster solution along with clustering in
general
you need to have cluster pairs such as
InventoryCluster/InventoryClusterHistory,
CusotmerCluster/CusotmerClusterHistory etc. if you do not want to loose
cluster
information in the history space.
Best regards,
Thomas

[Mich Geni]

Hi Thomas,

I think that it is bad programming practice to start carrying around

Post by Thomas
history information in
the Vertices and Edges (Sorry Mich - no offense). I'm not to crazy about
the timestamp solution
myself, but I thing that this is the best solution.

I dont' know why you are saying this?


Regards,


Mich

Post by Thomas
Hi Luca and Mich,
The original idea was that all Vertices and Edges have a mandatory
timestamp field of cause,
and the idea was also to treat Vertices and Edges the same (with a
mandatory timestamp ONLY).
I think that it is bad programming practice to start carrying around
history information in
the Vertices and Edges (Sorry Mich - no offense). I'm not to crazy about
the timestamp solution
myself, but I thing that this is the best solution.
How do we update the timestamp for a modified Vertex or Edge? Originally
I thought
that this would be done before an update (before the hook is called), but
we could of cause
automate it and have the timestamp updated inside the hook. If we do it
outside the hook
however, the posted code will take care of copying the old timestamp to
the history version
of the Vertex or Edge, but I agree, why do it manually if we can automate
it inside the hook.
There is however one important reason for updating the timestamp from
outside the hook, and
that is if we need to keep working with the document which may cause
inconsistency between
the version in memory and the version in storage (Maybe - I don't know if
this could become a
problem).
In the timestamp solution we need however to take into account when a
Vertex or Edge
is deleted. We need a non-mandatory field e.g. deletetimestamp if we do
not want to loose
history information about when a document was deleted. The original
timestamp only tells
us when a document was last updated (this is saved to History) but since
we are deleting
the document we only have the time for last modification, not the
deletetimestamp. Thus
we need to add a deletetimestamp to the last History document before
deletion.
If you think it through, the simplicity of the proposed
timestamp/History cluster solution
is very beautiful. We can always search the history cluster and find out
exactly how a Vertex
or Edge was connected and what the field values were at any given time.
Unless you are frequently working specifically with History I do not
think that we should
store other information than a simple mandatory timestamp. Note that no
"active" Vertex or
Edge (i.e. not in the History cluster) will contain the deletetimestamp.
Only a mandatory
timestamp indicating when it was created.
By the way Luca, I think it would be really cool to create a wiki page.
Can you give me an idea where to start :-)
Note: If you use the History cluster solution along with clustering in
general
you need to have cluster pairs such as
InventoryCluster/InventoryClusterHistory,
CusotmerCluster/CusotmerClusterHistory etc. if you do not want to loose
cluster
information in the history space.
Best regards,
Thomas

[Thomas]

Hi Mich,

Maybe I put that wrong, but the only point that I tried to state is that it
is not good practice
to couple tings. To decouple Vertices and Edges from a history solution our
primary goal
should be to omit any history realted data in any documents. Don't get me
wrong, I actually
thing that your solution could be very valuable if youe ned to operate
extensively in the
history space, but your solution needs such a justification and should not
be implemented
if it can be avoided. Just to be clear, You do mean that we should add
additional fields to
documents in order to implement your proposed solution right.

I am sorry if I offended you Mich.

Best regards,
Thomas

[Mich Geni]

Hi Thomas,

I think it is much simple than this.


- We need meta-data for storing history of a vertex/edge/document. And
that needs to be stored somewhere
- One place is to store it in separate classes and relate/link that
information to the nodes (with history). But what about:
- the queries (how we will query such meta-info)
- the additional lookups etc.?
- The other way is to store such meta-data along with the
vertices/edges/documents. Which appears good because:
- Your queries will be much simpler then
- No additional lookups to foreign classes/nodes, just for history
meta-data
- It doesn't matter if you add one or two fields for history, if it
makes your life easier
- Decoupling is good, when coupling introduce complexities. In this
particular case, a set of few fields added to each document will not bring
devils from hell to cast fire on us.
- By the way, shouldn't meta-data (e.g. history) be stored with a
document itself? rather than in a separate place, where you will have to
make sure about the consistency of both the document and its meta-data.

About offending, I was not offended, but I wanted to be sure that I made my
point clear. ;)


Regards,

Mich

Post by Thomas
Hi Mich,
Maybe I put that wrong, but the only point that I tried to state is that
it is not good practice
to couple tings. To decouple Vertices and Edges from a history solution
our primary goal
should be to omit any history realted data in any documents. Don't get me
wrong, I actually
thing that your solution could be very valuable if youe ned to operate
extensively in the
history space, but your solution needs such a justification and should not
be implemented
if it can be avoided. Just to be clear, You do mean that we should add
additional fields to
documents in order to implement your proposed solution right.
I am sorry if I offended you Mich.
Best regards,
Thomas
Hi Mich,
Maybe I put that wrong, but the only point that I tried to state is that
it is not good practice
to couple tings. To decouple Vertices and Edges from a history solution
our primary goal
should be to omit any history realted data in any documents. Don't get me
wrong, I actually
thing that your solution could be very valuable if youe ned to operate
extensively in the
history space, but your solution needs such a justification and should not
be implemented
if it can be avoided. Just to be clear, You do mean that we should add
additional fields to
documents in order to implement your proposed solution right.
I am sorry if I offended you Mich.
Best regards,
Thomas

[Thomas]

Hi Mich,

Your point:
We need meta-data for storing history of a vertex/edge/document. And
that needs to be stored somewhere.

What I am saying is that this is excactly what we should avoid (in the
History cluster solution), but I'm not
saying that the History cluster solution is the best or right solution.

We can go down three roads.

1. Use a History cluster to create a graph containing all information (as
documents) current as well as history.
2. Avoid using clusters to implement the solution by adding history
meta-data to documents.
3. Hybrid solution.

I'm really not sure which is better Mich. I think that the advantage of a
cluster solution is that everything is
ODocuments in the graph. It's a graph and nothing more (yea - except for
the timestamp that is). We can
search the graph with a minimum of attention to fields and history. Again
it depends on the context I guess.

Best regards,
Thomas

[Thomas]

Hi Luca,

I appreciate that you have created a wiki and I am very eager to write a
good use case,
and I hope that Mich can contribute to the wiki as well providing a great
use case for his
proposed solution :-)

Luca im am currently working on the history solution an have come up with a
fully transparent
timestamp wich is created when you create a vertex or edge. If you try to
create a timestamp
field when creating a document an OValidationException will be thrown, and
if you try to modify
a timestamp an exception is thrown as well. The goal here is to make the
history solution as
invisible as possible.

Since my work on the History solution is work related I have consulted my
boss and told him that
I wish to contribute to a wiki by providing our solution to the public. He
thinks that it is a great idea
but naturally he needs to cunsult with the boead of directors before he can
give me the go. But
I'm sure that it will be OK. Together we can reach new hights :-)

Tomorrow I'm going on a three weeks vaication and my boss told me that we
can take it from there.
So Luca I have to get back to you as soon as I know what I can and can't
do. Anyway for now I am
going on a holiday with my girlfriend.

Have a nice summer everyone.

Best regards,
Thomas

[Thomas]

Hi everyone,

Back from holiday :-)

Luca I have spoken to my boss and he still thinks that it is a great idea
if we contribute to the community by adding use cases to the wiki you
created,
however before we do so we have to finish the first draft of the history
solution since it is fundamental in our future transition to OrientDB.
Until then
the community can follow the development here or at least get some ideas
here.

We have however run into a little problem when vertices are deleted (the
problem is probably also there with edges).

When a vertex is deleted this is what happens:

All edges connected to the vertex are deleted and the pointers to
the edges in the opposite vertices are removed (nice cleanup by the way)

The steps are basically:

- Take all in and out edges from the vertex to delete
- Go to the other vertices and remove the edge pointer (ORID) connecting
the vertices to the vertex to be deleted
- Delete the edge documents
- Finally delete the vertex

THE PROBLEM IS:

in and out fields are removed/deleted in vertices before the hook is
called. Thus we have no way of copying the vertices (containing in and out
fields) correctly to the history space.
WHEN THE HOOK IS CALLED THE in AND out FIELDS HAVE ALREADY BEEN REMOVED
AND WE CAN'T ACCESS THE FIELDS USING THE doc.getOriginalValue EITHER.

QUESTION:

Shouldn't we call the hook before finally deleting anything?

======================================================================================================
*Simple example (written in scala but should be easily readable by java
programmers) that create two vertices, connects them
with an edge and finally deletes the first vertex.

* val database = new OGraphDatabase(path)
database.open(username, password)
database.registerHook(this) // this class extends the hook class

val odoc1: ODocument = database.createVertex("MyVertexClassName1")
odoc1.field("someFieldName", "someFieldValue")
odoc1.save(defaultCluster)

val odoc2: ODocument = database.createVertex("MyVertexClassName2")
odoc2.field("someFieldName", "someFieldValue")
odoc2.save(defaultCluster)

val edgeDoc: ODocument = database.createEdge(odoc1, odoc2,
"MyEdgeClassName")
edgeDoc.save(defaultCluster)

database.removeVertex(odoc1) *// This is where the problems start**(see below)
**
*
======================================================================================================
*Code reference from OGraphDatabase.java (look at the comments)*

*public void removeVertex**(final ODocument iVertex) {*
final boolean safeMode = beginBlock();

try {
ODocument otherVertex;
Set<ODocument> otherEdges;

*// REMOVE OUT EDGES*
Set<ODocument> edges = iVertex.field(VERTEX_FIELD_OUT);
if (edges != null) {
for (ODocument edge : edges) {
if (edge != null) {
otherVertex = edge.field(EDGE_FIELD_IN);
if (otherVertex != null) {
otherEdges = otherVertex.field(VERTEX_FIELD_IN);
if (otherEdges != null && otherEdges.remove(edge))
save(otherVertex); *// onRecordBeforeUpdate is called after
this**. Vertex will be copied ** (before it is updated) **to history, but
we CAN NOT ACCESS in/out** in the hook*.
}
delete(edge); *// onRecordBeforeDelete is called after this.
Edge** will be copied (before it is deleted) to history. NOTE that we DO
HAVE ACCESS to in/out in the hook for edges.*
}
}
edges.clear();
iVertex.field(VERTEX_FIELD_OUT, edges);
}

*// REMOVE IN EDGES*
edges = iVertex.field(VERTEX_FIELD_IN);
if (edges != null) {
for (ODocument edge : edges) {
if (edge != null) {
otherVertex = edge.field(EDGE_FIELD_OUT);
if (otherVertex != null) {
otherEdges = otherVertex.field(VERTEX_FIELD_OUT);
if (otherEdges != null && otherEdges.remove(edge))
save(otherVertex);
}
delete(edge);
}
}
edges.clear();
iVertex.field(VERTEX_FIELD_IN, edges);
}

*// DELETE VERTEX AS DOCUMENT*
delete(iVertex);* // onRecordBeforeDelete is called after this.
Vertex will be copied ** (before it is deleted) **to history**, but we **CAN
NOT ACCESS in/out** in the hook*.

commitBlock(safeMode);

} catch (RuntimeException e) {
rollbackBlock(safeMode);
throw e;
}
* }*
======================================================================================================

Somehow the in/out fields are deleted before the hook is called (when it
comes to vertices)

By the way Mich (if you are still there) I have given it some thought and I
actually think that you are absolutely right with your linked list idea. In
order
to search from history cluster to current cluster and the other way around
we need something like a hist field in the current cluster nodes (vertices
and edges)
and we need a parent field in the history nodes.

Luca (or someone) please help

Thanks

Best regards
Thomas

--

[Thomas]

I have added three comments to the code reference from OGraphDatabase.java
that may give an idea on how to solve the problem. I hope :-)

The hard one to solve i think is the

if (otherEdges != null && otherEdges.remove(edge))
save(otherVertex);

in the *// REMOVE OUT EDGES* and in the *// REMOVE IN EDGES*

We need remove the edge before saving the otherVertex but we need to be
aware of what was deleted when onRecordBeforeUpdate is called. Hmmmm...

======================================================================================================
*Code reference from OGraphDatabase.java*

*public void removeVertex(final ODocument iVertex) {*
final boolean safeMode = beginBlock();

try {
ODocument otherVertex;
Set<ODocument> otherEdges;

*// REMOVE OUT EDGES*
Set<ODocument> edges = iVertex.field(VERTEX_FIELD_OUT);
if (edges != null) {
for (ODocument edge : edges) {
if (edge != null) {
otherVertex = edge.field(EDGE_FIELD_IN);
if (otherVertex != null) {
otherEdges = otherVertex.field(VERTEX_FIELD_IN);
if (otherEdges != null && otherEdges.remove(edge)) *// SINCE
WE REMOVE THE EDGE POINTER IN otherVertex (remember that otherEdges is the
in-field from otherVertex) HERE WE DO NOT HAVE ACCESS TO IN
onRecordBeforeUpdate (see next line -- save(otherVertex)) *
save(otherVertex);* // onRecordBeforeUpdate is called after
this. Vertex will be copied (before it is updated) to history, but we CAN
NOT ACCESS in/out in the hook.*
}
delete(edge);* // onRecordBeforeDelete is called after this.
Edge will be copied (before it is deleted) to history. NOTE that we DO HAVE
ACCESS to in/out in the hook for edges.*
}
}
edges.clear(); *// WHY CLEAR THE OUT FIELD HERE - THIS WILL
PROHIBIT OS FROM USING IT IN THE HOOK. I CAN'T SEE THAT WE USE THE OUT
FIELD AGAIN BEFORE THE VERTEX IS DELETED (see last red comment --
delete(iVertex))*
iVertex.field(VERTEX_FIELD_OUT, edges); *// HERE WE CLEAR THE OUT
FIELD IN THE VERTEX TO DELETE. THUS WE DO NOT HAVE ACCESS TO IT IN
onRecordBeforeDelete (see last red comment -- delete(iVertex))*
}

*// REMOVE IN EDGES*
edges = iVertex.field(VERTEX_FIELD_IN);
if (edges != null) {
for (ODocument edge : edges) {
if (edge != null) {
otherVertex = edge.field(EDGE_FIELD_OUT);
if (otherVertex != null) {
otherEdges = otherVertex.field(VERTEX_FIELD_OUT);
if (otherEdges != null && otherEdges.remove(edge))
save(otherVertex);
}
delete(edge);
}
}
edges.clear();
iVertex.field(VERTEX_FIELD_IN, edges);
}

// DELETE VERTEX AS DOCUMENT
delete(iVertex);* // onRecordBeforeDelete is called after this.
Vertex will be copied (before it is deleted) to history, but we CAN NOT
ACCESS in/out in the hook.
*
commitBlock(safeMode);

} catch (RuntimeException e) {
rollbackBlock(safeMode);
throw e;
}
* }*
======================================================================================================

Best regards
Thomas

--

[Luca Garulli]

Hi Thomas,
can you write a patch that works for your case and the entire test suite by
changing the order of the code inside the removeVertex() ? (run: ant clean
test)

Post by Thomas
I have added three comments to the code reference from OGraphDatabase.java
that may give an idea on how to solve the problem. I hope :-)
The hard one to solve i think is the
if (otherEdges != null && otherEdges.remove(edge))
save(otherVertex);
in the *// REMOVE OUT EDGES* and in the *// REMOVE IN EDGES*
We need remove the edge before saving the otherVertex but we need to be
aware of what was deleted when onRecordBeforeUpdate is called. Hmmmm...
======================================================================================================
*Code reference from OGraphDatabase.java*
*public void removeVertex(final ODocument iVertex) {*
final boolean safeMode = beginBlock();
try {
ODocument otherVertex;
Set<ODocument> otherEdges;
*// REMOVE OUT EDGES*
Set<ODocument> edges = iVertex.field(VERTEX_FIELD_OUT);
if (edges != null) {
for (ODocument edge : edges) {
if (edge != null) {
otherVertex = edge.field(EDGE_FIELD_IN);
if (otherVertex != null) {
otherEdges = otherVertex.field(VERTEX_FIELD_IN);
if (otherEdges != null && otherEdges.remove(edge)) *//
SINCE WE REMOVE THE EDGE POINTER IN otherVertex (remember that otherEdges
is the in-field from otherVertex) HERE WE DO NOT HAVE ACCESS TO IN
onRecordBeforeUpdate (see next line -- save(otherVertex)) *
save(otherVertex);* // onRecordBeforeUpdate is called
after this. Vertex will be copied (before it is updated) to history, but we
CAN NOT ACCESS in/out in the hook.*
}
delete(edge);* // onRecordBeforeDelete is called after this.
Edge will be copied (before it is deleted) to history. NOTE that we DO HAVE
ACCESS to in/out in the hook for edges.*
}
}
edges.clear(); *// WHY CLEAR THE OUT FIELD HERE - THIS WILL
PROHIBIT OS FROM USING IT IN THE HOOK. I CAN'T SEE THAT WE USE THE OUT
FIELD AGAIN BEFORE THE VERTEX IS DELETED (see last red comment --
delete(iVertex))*
iVertex.field(VERTEX_FIELD_OUT, edges); *// HERE WE CLEAR THE OUT
FIELD IN THE VERTEX TO DELETE. THUS WE DO NOT HAVE ACCESS TO IT IN
onRecordBeforeDelete (see last red comment -- delete(iVertex))*
}
*// REMOVE IN EDGES*
edges = iVertex.field(VERTEX_FIELD_IN);
if (edges != null) {
for (ODocument edge : edges) {
if (edge != null) {
otherVertex = edge.field(EDGE_FIELD_OUT);
if (otherVertex != null) {
otherEdges = otherVertex.field(VERTEX_FIELD_OUT);
if (otherEdges != null && otherEdges.remove(edge))
save(otherVertex);
}
delete(edge);
}
}
edges.clear();
iVertex.field(VERTEX_FIELD_IN, edges);
}
// DELETE VERTEX AS DOCUMENT
delete(iVertex);* // onRecordBeforeDelete is called after this.
Vertex will be copied (before it is deleted) to history, but we CAN NOT
ACCESS in/out in the hook.
*
commitBlock(safeMode);
} catch (RuntimeException e) {
rollbackBlock(safeMode);
throw e;
}
* }*
======================================================================================================
Best regards
Thomas
--

--

[Thomas]

Hi Luca,

Nice to hear from you.

Yes I will give it a go. I'll get back as soon as I have a solution [ or
any problems ;-) ] and have tested the solution with the entire test suite.

Best regards
Thomas

--

[Thomas]

Hi,

The first patch is build from orientdb-graphed-1.1.0 sources (latest stable
release).

This is a fix to the simple problem.

The following have been commented out in *// REMOVE OUT EDGES*

*// edges.clear();
//iVertex.field(VERTEX_FIELD_OUT, edges); *

and the following have been commented out in *// REMOVE IN EDGES*

*// edges.clear();
// iVertex.field(VERTEX_FIELD_IN, edges);

*The changes have been compiled and all tested using:

ant clean test

as you prescribed Luca

All tests pass and we can now access in/out fields in the hook :-)

======================================================================================================
*Code reference from the changed OGraphDatabase.java*

*public void removeVertex(final ODocument iVertex) {*
final boolean safeMode = beginBlock();

try {
ODocument otherVertex;
Set<ODocument> otherEdges;

*// REMOVE OUT EDGES*
Set<ODocument> edges = iVertex.field(VERTEX_FIELD_OUT);
if (edges != null) {
for (ODocument edge : edges) {
if (edge != null) {
otherVertex = edge.field(EDGE_FIELD_IN);
if (otherVertex != null) {
otherEdges = otherVertex.field(VERTEX_FIELD_IN);
if (otherEdges != null && otherEdges.remove(edge))
save(otherVertex);
}
delete(edge);**
}
}
*// edges.clear();
//iVertex.field(VERTEX_FIELD_OUT, edges); *
}

*// REMOVE IN EDGES*
edges = iVertex.field(VERTEX_FIELD_IN);
if (edges != null) {
for (ODocument edge : edges) {
if (edge != null) {
otherVertex = edge.field(EDGE_FIELD_OUT);
if (otherVertex != null) {
otherEdges = otherVertex.field(VERTEX_FIELD_OUT);
if (otherEdges != null && otherEdges.remove(edge))
save(otherVertex);
}
delete(edge);
}
}
*// edges.clear();
// iVertex.field(VERTEX_FIELD_IN, edges);*
}

*// DELETE VERTEX AS DOCUMENT*
delete(iVertex);* // **WE CAN NOW ACCESS in/out IN THE HOOKS **onRecordBeforeDelete

*
commitBlock(safeMode);

} catch (RuntimeException e) {
rollbackBlock(safeMode);
throw e;
}
* }*
======================================================================================================

Now on to the tricky problem (I think)

Best regards
Thomas

--

[Luca Garulli]

Hi,
thanks, fixed in SVN r6449.

Good luck for the rest :-) !

Post by Thomas
Hi,
The first patch is build from orientdb-graphed-1.1.0 sources (latest
stable release).
This is a fix to the simple problem.
The following have been commented out in *// REMOVE OUT EDGES*
*// edges.clear();
//iVertex.field(VERTEX_FIELD_OUT, edges); *
and the following have been commented out in *// REMOVE IN EDGES*
*// edges.clear();
// iVertex.field(VERTEX_FIELD_IN, edges);
ant clean test
as you prescribed Luca
All tests pass and we can now access in/out fields in the hook :-)
======================================================================================================
*Code reference from the changed OGraphDatabase.java*
*public void removeVertex(final ODocument iVertex) {*
final boolean safeMode = beginBlock();
try {
ODocument otherVertex;
Set<ODocument> otherEdges;
*// REMOVE OUT EDGES*
Set<ODocument> edges = iVertex.field(VERTEX_FIELD_**OUT);
if (edges != null) {
for (ODocument edge : edges) {
if (edge != null) {
otherVertex = edge.field(EDGE_FIELD_IN);
if (otherVertex != null) {
otherEdges = otherVertex.field(VERTEX_**FIELD_IN);
if (otherEdges != null && otherEdges.remove(edge))
save(otherVertex);
}
delete(edge);**
}
}
*// edges.clear();
//iVertex.field(VERTEX_FIELD_OUT, edges); *
}
*// REMOVE IN EDGES*
edges = iVertex.field(VERTEX_FIELD_IN)**;
if (edges != null) {
for (ODocument edge : edges) {
if (edge != null) {
otherVertex = edge.field(EDGE_FIELD_OUT);
if (otherVertex != null) {
otherEdges = otherVertex.field(VERTEX_**FIELD_OUT);
if (otherEdges != null && otherEdges.remove(edge))
save(otherVertex);
}
delete(edge);
}
}
*// edges.clear();
// iVertex.field(VERTEX_FIELD_IN, edges);*
}
*// DELETE VERTEX AS DOCUMENT*
delete(iVertex);* // **WE CAN NOW ACCESS in/out IN THE HOOKS **onRecordBeforeDelete
*
commitBlock(safeMode);
} catch (RuntimeException e) {
rollbackBlock(safeMode);
throw e;
}
* }*
==============================**==============================**
==============================**============
Now on to the tricky problem (I think)
Best regards
Thomas
--

--

[Thomas]

Hi Luca,

I'm not sure how to solve the issue with:

=================================================================
if (otherVertex != null) {
otherEdges = otherVertex.field(VERTEX_FIELD_OUT); // same for
VERTEX_FIELD_IN
if (otherEdges != null && otherEdges.remove(edge))
save(otherVertex);
}
delete(edge);
=================================================================

To be able to obtain IN and OUT fields before they are deleted, we need to
do something like the following:

=================================================================
if (otherVertex != null) {
otherEdges = otherVertex.field(VERTEX_FIELD_OUT); // same for
VERTEX_FIELD_IN
*otherVertex.setOriginalValues**(**VERTEX_FIELD_OUT**, otherEdges)* *//
Copy otherEdges to the _fieldOriginalValues Map of otherVertex before we
remove the edge form otherEdges*
if (otherEdges != null && otherEdges.remove(edge))
save(otherVertex);
}
delete(edge);
=================================================================

or we need a function like *otherVertex.removeEdge**(**VERTEX_FIELD_OUT**,
edge) *that take care of the copy -> remove -> save before delete (which
call the hook)

This would of cause require a *setOriginalValues* function in
ODocument.java which kind of ruin the fact that _fieldOriginalValues is a
protected field (or we could make _fieldOriginalValues public) or
a *removeEdge* function in ODocument.java which would be dangerous because
we do not want users to tamper with the IN and OUT Set of ODocument.

These are not very attractive solutions, but I don't really see any other
way.

Does anyone have any ideas?

Best regards
Thomas

--

[Thomas]

I'm back

Hi Luca,

I have now come up with a solution that does the job i.e. enables us to
access old fields
inside the hook. The solution use a function inside ODocument.java that
basically copies
every ORecordId from the OMVRBTreeRIDSet of in or out to a new
OMVRBTreeRIDSet which is
stored in _fieldOriginalValues. The removeVertex and removeEdge is modified
to call this
function before any remove calls. The code is shown below.

I have run the entire test suite as earlier and all test pass.

Also i have run some initial history tests and we are now able
to obtain the original value of in and out inside the hook when
a Vertex or Edge is remove/deleted.

Before we continue I should note that we after some heavy discussions we
have concluded that
whenever we delete a vertex all affected vertices must have their pointer
to this vertex
deleted. Thus we modify all affected vertices and therefore must copy them
to the history
space. The same concept goes for edges. Thus:

1. Delete vertex: The vertex must be copied to history along with all in
and out edges (which
are also to be deleted) and we also copy the vertex (which is only updated)
at the other end
of all affected edges. These nodes must point to each other in the history
space because that
is what they did at the time when we executed the delete vertex operation.
If we simply copy
the vertices to history whey will point to edges that are non-existent
after the delete
operation.

2. Delete edge: The same principle goes when deleting edges. We need to
copy (before we delete)
the edge to history along with the two connected vertices (which is only
updated) if we want to
avoid dead pointers (i.e. if the vertices are later deleted) from the edge
in history to the
vertices in the default cluster.

NOW HERE IS THE SHOWSTOPPER FOR THE HOOK SOLUTION:

We can't use a hook if we want to avoid dead links from history space to
default space after
having made a delete operation. If we e.g. delete a vertex V1 then all
affected vertices must
have their pointer to V1 deleted. This will call the update hook with
these vertices (one at
the time of cause). Inside the update hook we do not know if the hook was
called in the context
of a delete (in which case it should point to an edge in the history
space) or in the context
of a field update (in which case it should point to an edge in the
default space). We need to
solve this at a higher level than at the hook level.

We will now proceed to a solution that either extends the OGraphDatabase
and ODocument or wraps
the graph database such that we can use it with history.

In the future it would be nice with an embedded solution such that we can
just call a
constructor (see below) with a flag to enable history.

public OGraphDatabase(final String iURL) {
this.OGraphDatabase(iURL, false);
}

public OGraphDatabase(final String iURL, boolean iWithHistory) {
withHistory = iWithHistory; // To be used in the imbedded solution
}

==================================================================================
Added functions in ODocument.java
----------------------------------

* /**
* Clones the ORecordId's of the OMVRBTreeRIDSet found in
* the in field and copies the new tree to _fieldOriginalValues.
*/*
public void cloneInToOriginalValue() {
cloneInOrOutToFieldOriginalValues("in");
}

*/**
* Clones the ORecordId's of the OMVRBTreeRIDSet found in
* the out field and copies the new tree to _fieldOriginalValues.
*/*
public void cloneOutToOriginalValue() {
cloneInOrOutToFieldOriginalValues("out");
}

protected void cloneInOrOutToFieldOriginalValues(String iFieldName) {
OMVRBTreeRIDSet set = getClonedInOrOutField(iFieldName);
if(set != null) {
if (_fieldOriginalValues == null)
_fieldOriginalValues = new HashMap<String, Object>();

_fieldOriginalValues.put(iFieldName, set);
}
}

protected OMVRBTreeRIDSet getClonedInOrOutField(String iFieldName) {
if (_fieldValues != null && _fieldValues.containsKey(iFieldName)) {
OMVRBTreeRIDSet set = new OMVRBTreeRIDSet();

for(OIdentifiable doc: ((OMVRBTreeRIDSet) field(iFieldName))) {
set.add(doc.getIdentity());
}
return set;
}
else return null;
}

==================================================================================


==================================================================================
Modified functions in OGraphDatabase.java
------------------------------------------

public void removeVertex(final ODocument iVertex) {
final boolean safeMode = beginBlock();

try {
ODocument otherVertex;
Set<ODocument> otherEdges;

*// REMOVE OUT EDGES*
Set<ODocument> edges = iVertex.field(VERTEX_FIELD_OUT);
if (edges != null) {
for (ODocument edge : edges) {
if (edge != null) {
otherVertex = edge.field(EDGE_FIELD_IN);
if (otherVertex != null) {
otherEdges = otherVertex.field(VERTEX_FIELD_IN);
*otherVertex.cloneInToOriginalValue();*
if (otherEdges != null && otherEdges.remove(edge))
save(otherVertex);
}
delete(edge);
}
}
}

*// REMOVE IN EDGES*
edges = iVertex.field(VERTEX_FIELD_IN);
if (edges != null) {
for (ODocument edge : edges) {
if (edge != null) {
otherVertex = edge.field(EDGE_FIELD_OUT);
if (otherVertex != null) {
otherEdges = otherVertex.field(VERTEX_FIELD_OUT);
*otherVertex.cloneOutToOriginalValue();*
if (otherEdges != null && otherEdges.remove(edge))
save(otherVertex);
}
delete(edge);
}
}
}

*// DELETE VERTEX AS DOCUMENT*
delete(iVertex);

commitBlock(safeMode);

} catch (RuntimeException e) {
rollbackBlock(safeMode);
throw e;
}
}

@SuppressWarnings("unchecked")
public void removeEdge(final ODocument iEdge) {
final boolean safeMode = beginBlock();

try {
final ODocument outVertex = iEdge.field(EDGE_FIELD_OUT);
if (outVertex != null) {
final Set<ODocument> out = ((Set<ODocument>)
outVertex.field(VERTEX_FIELD_OUT));
*outVertex.cloneOutToOriginalValue();*
if (out != null && out.remove(iEdge))
save(outVertex);
}

final ODocument inVertex = iEdge.field(EDGE_FIELD_IN);
if (inVertex != null) {
final Set<ODocument> in = ((Set<ODocument>)
inVertex.field(VERTEX_FIELD_IN));
* inVertex.cloneInToOriginalValue();*
if (in != null && in.remove(iEdge))
save(inVertex);
}

* // DELETE EDGE AS DOCUMENT*
delete(iEdge);

commitBlock(safeMode);

} catch (RuntimeException e) {
rollbackBlock(safeMode);
throw e;
}
}

==================================================================================

Note that when vertices are saved, they will be exactly the same as if
cloneInToOriginalValue or
cloneOutToOriginalValue were not called i.e. without any entries in the
_fieldOriginalValues.
We do not have to clear _fieldOriginalValues.

I'll be back – when we come up with a new non-hook solution ;-)

Best regards
Thomas

--

[Luca Garulli]

Hi Thomas,
thank you for such progress on this. Unfortunately I can't apply you patch
because ODocument is a more generic class and can't know the meaning of
"in" and "out" of Graph Database model. Furthermore WDYT about
letting OMVRBTreeRIDSet implement the track of changes like other
collections?

Will this fix your need? Is it enough?

Post by Thomas
I'm back
Hi Luca,
I have now come up with a solution that does the job i.e. enables us to
access old fields
inside the hook. The solution use a function inside ODocument.java that
basically copies
every ORecordId from the OMVRBTreeRIDSet of in or out to a new
OMVRBTreeRIDSet which is
stored in _fieldOriginalValues. The removeVertex and removeEdge is
modified to call this
function before any remove calls. The code is shown below.
I have run the entire test suite as earlier and all test pass.
Also i have run some initial history tests and we are now able
to obtain the original value of in and out inside the hook when
a Vertex or Edge is remove/deleted.
Before we continue I should note that we after some heavy discussions we
have concluded that
whenever we delete a vertex all affected vertices must have their pointer
to this vertex
deleted. Thus we modify all affected vertices and therefore must copy them
to the history
1. Delete vertex: The vertex must be copied to history along with all in
and out edges (which
are also to be deleted) and we also copy the vertex (which is only
updated) at the other end
of all affected edges. These nodes must point to each other in the history
space because that
is what they did at the time when we executed the delete vertex operation.
If we simply copy
the vertices to history whey will point to edges that are non-existent
after the delete
operation.
2. Delete edge: The same principle goes when deleting edges. We need to
copy (before we delete)
the edge to history along with the two connected vertices (which is only
updated) if we want to
avoid dead pointers (i.e. if the vertices are later deleted) from the edge
in history to the
vertices in the default cluster.
We can't use a hook if we want to avoid dead links from history space to
default space after
having made a delete operation. If we e.g. delete a vertex V1 then all
affected vertices must
have their pointer to V1 deleted. This will call the update hook with
these vertices (one at
the time of cause). Inside the update hook we do not know if the hook
was called in the context
of a delete (in which case it should point to an edge in the history
space) or in the context
of a field update (in which case it should point to an edge in the
default space). We need to
solve this at a higher level than at the hook level.
We will now proceed to a solution that either extends the OGraphDatabase
and ODocument or wraps
the graph database such that we can use it with history.
In the future it would be nice with an embedded solution such that we can
just call a
constructor (see below) with a flag to enable history.
public OGraphDatabase(final String iURL) {
this.OGraphDatabase(iURL, false);
}
public OGraphDatabase(final String iURL, boolean iWithHistory) {
withHistory = iWithHistory; // To be used in the imbedded solution
}
==================================================================================
Added functions in ODocument.java
----------------------------------
* /**
* Clones the ORecordId's of the OMVRBTreeRIDSet found in
* the in field and copies the new tree to _fieldOriginalValues.
*/*
public void cloneInToOriginalValue() {
cloneInOrOutToFieldOriginalValues("in");
}
*/**
* Clones the ORecordId's of the OMVRBTreeRIDSet found in
* the out field and copies the new tree to _fieldOriginalValues.
*/*
public void cloneOutToOriginalValue() {
cloneInOrOutToFieldOriginalValues("out");
}
protected void cloneInOrOutToFieldOriginalValues(String iFieldName) {
OMVRBTreeRIDSet set = getClonedInOrOutField(iFieldName);
if(set != null) {
if (_fieldOriginalValues == null)
_fieldOriginalValues = new HashMap<String, Object>();
_fieldOriginalValues.put(iFieldName, set);
}
}
protected OMVRBTreeRIDSet getClonedInOrOutField(String iFieldName) {
if (_fieldValues != null && _fieldValues.containsKey(iFieldName)) {
OMVRBTreeRIDSet set = new OMVRBTreeRIDSet();
for(OIdentifiable doc: ((OMVRBTreeRIDSet) field(iFieldName))) {
set.add(doc.getIdentity());
}
return set;
}
else return null;
}
==================================================================================
==================================================================================
Modified functions in OGraphDatabase.java
------------------------------------------
public void removeVertex(final ODocument iVertex) {
final boolean safeMode = beginBlock();
try {
ODocument otherVertex;
Set<ODocument> otherEdges;
*// REMOVE OUT EDGES*
Set<ODocument> edges = iVertex.field(VERTEX_FIELD_OUT);
if (edges != null) {
for (ODocument edge : edges) {
if (edge != null) {
otherVertex = edge.field(EDGE_FIELD_IN);
if (otherVertex != null) {
otherEdges = otherVertex.field(VERTEX_FIELD_IN);
*otherVertex.cloneInToOriginalValue();*
if (otherEdges != null && otherEdges.remove(edge))
save(otherVertex);
}
delete(edge);
}
}
}
*// REMOVE IN EDGES*
edges = iVertex.field(VERTEX_FIELD_IN);
if (edges != null) {
for (ODocument edge : edges) {
if (edge != null) {
otherVertex = edge.field(EDGE_FIELD_OUT);
if (otherVertex != null) {
otherEdges = otherVertex.field(VERTEX_FIELD_OUT);
*otherVertex.cloneOutToOriginalValue();*
if (otherEdges != null && otherEdges.remove(edge))
save(otherVertex);
}
delete(edge);
}
}
}
*// DELETE VERTEX AS DOCUMENT*
delete(iVertex);
commitBlock(safeMode);
} catch (RuntimeException e) {
rollbackBlock(safeMode);
throw e;
}
}
@SuppressWarnings("unchecked")
public void removeEdge(final ODocument iEdge) {
final boolean safeMode = beginBlock();
try {
final ODocument outVertex = iEdge.field(EDGE_FIELD_OUT);
if (outVertex != null) {
final Set<ODocument> out = ((Set<ODocument>)
outVertex.field(VERTEX_FIELD_OUT));
*outVertex.cloneOutToOriginalValue();*
if (out != null && out.remove(iEdge))
save(outVertex);
}
final ODocument inVertex = iEdge.field(EDGE_FIELD_IN);
if (inVertex != null) {
final Set<ODocument> in = ((Set<ODocument>)
inVertex.field(VERTEX_FIELD_IN));
* inVertex.cloneInToOriginalValue();*
if (in != null && in.remove(iEdge))
save(inVertex);
}
* // DELETE EDGE AS DOCUMENT*
delete(iEdge);
commitBlock(safeMode);
} catch (RuntimeException e) {
rollbackBlock(safeMode);
throw e;
}
}
==================================================================================
Note that when vertices are saved, they will be exactly the same as if
cloneInToOriginalValue or
cloneOutToOriginalValue were not called i.e. without any entries in the
_fieldOriginalValues.
We do not have to clear _fieldOriginalValues.
I'll be back – when we come up with a new non-hook solution ;-)
Best regards
Thomas
--

--

[Luca Garulli]

Hi Thomas,
some hours ago I committed in SVN trunk the new OMVRBTreeRIDSet that
supports trackable changes. Could it fix your problem? Could you run some
tests against it?

Post by Luca Garulli
Hi Thomas,
thank you for such progress on this. Unfortunately I can't apply you patch
because ODocument is a more generic class and can't know the meaning of
"in" and "out" of Graph Database model. Furthermore WDYT about
letting OMVRBTreeRIDSet implement the track of changes like other
collections?
Will this fix your need? Is it enough?

Post by Thomas
I'm back
Hi Luca,
I have now come up with a solution that does the job i.e. enables us to
access old fields
inside the hook. The solution use a function inside ODocument.java that
basically copies
every ORecordId from the OMVRBTreeRIDSet of in or out to a new
OMVRBTreeRIDSet which is
stored in _fieldOriginalValues. The removeVertex and removeEdge is
modified to call this
function before any remove calls. The code is shown below.
I have run the entire test suite as earlier and all test pass.
Also i have run some initial history tests and we are now able
to obtain the original value of in and out inside the hook when
a Vertex or Edge is remove/deleted.
Before we continue I should note that we after some heavy discussions we
have concluded that
whenever we delete a vertex all affected vertices must have their pointer
to this vertex
deleted. Thus we modify all affected vertices and therefore must copy
them to the history
1. Delete vertex: The vertex must be copied to history along with all in
and out edges (which
are also to be deleted) and we also copy the vertex (which is only
updated) at the other end
of all affected edges. These nodes must point to each other in the
history space because that
is what they did at the time when we executed the delete vertex
operation. If we simply copy
the vertices to history whey will point to edges that are non-existent
after the delete
operation.
2. Delete edge: The same principle goes when deleting edges. We need to
copy (before we delete)
the edge to history along with the two connected vertices (which is only
updated) if we want to
avoid dead pointers (i.e. if the vertices are later deleted) from the
edge in history to the
vertices in the default cluster.
We can't use a hook if we want to avoid dead links from history space
to default space after
having made a delete operation. If we e.g. delete a vertex V1 then all
affected vertices must
have their pointer to V1 deleted. This will call the update hook with
these vertices (one at
the time of cause). Inside the update hook we do not know if the hook
was called in the context
of a delete (in which case it should point to an edge in the history
space) or in the context
of a field update (in which case it should point to an edge in the
default space). We need to
solve this at a higher level than at the hook level.
We will now proceed to a solution that either extends the OGraphDatabase
and ODocument or wraps
the graph database such that we can use it with history.
In the future it would be nice with an embedded solution such that we can
just call a
constructor (see below) with a flag to enable history.
public OGraphDatabase(final String iURL) {
this.OGraphDatabase(iURL, false);
}
public OGraphDatabase(final String iURL, boolean iWithHistory) {
withHistory = iWithHistory; // To be used in the imbedded solution
}
==================================================================================
Added functions in ODocument.java
----------------------------------
* /**
* Clones the ORecordId's of the OMVRBTreeRIDSet found in
* the in field and copies the new tree to _fieldOriginalValues.
*/*
public void cloneInToOriginalValue() {
cloneInOrOutToFieldOriginalValues("in");
}
*/**
* Clones the ORecordId's of the OMVRBTreeRIDSet found in
* the out field and copies the new tree to _fieldOriginalValues.
*/*
public void cloneOutToOriginalValue() {
cloneInOrOutToFieldOriginalValues("out");
}
protected void cloneInOrOutToFieldOriginalValues(String iFieldName) {
OMVRBTreeRIDSet set = getClonedInOrOutField(iFieldName);
if(set != null) {
if (_fieldOriginalValues == null)
_fieldOriginalValues = new HashMap<String, Object>();
_fieldOriginalValues.put(iFieldName, set);
}
}
protected OMVRBTreeRIDSet getClonedInOrOutField(String iFieldName) {
if (_fieldValues != null && _fieldValues.containsKey(iFieldName)) {
OMVRBTreeRIDSet set = new OMVRBTreeRIDSet();
for(OIdentifiable doc: ((OMVRBTreeRIDSet) field(iFieldName))) {
set.add(doc.getIdentity());
}
return set;
}
else return null;
}
==================================================================================
==================================================================================
Modified functions in OGraphDatabase.java
------------------------------------------
public void removeVertex(final ODocument iVertex) {
final boolean safeMode = beginBlock();
try {
ODocument otherVertex;
Set<ODocument> otherEdges;
*// REMOVE OUT EDGES*
Set<ODocument> edges = iVertex.field(VERTEX_FIELD_OUT);
if (edges != null) {
for (ODocument edge : edges) {
if (edge != null) {
otherVertex = edge.field(EDGE_FIELD_IN);
if (otherVertex != null) {
otherEdges = otherVertex.field(VERTEX_FIELD_IN);
*otherVertex.cloneInToOriginalValue();*
if (otherEdges != null && otherEdges.remove(edge))
save(otherVertex);
}
delete(edge);
}
}
}
*// REMOVE IN EDGES*
edges = iVertex.field(VERTEX_FIELD_IN);
if (edges != null) {
for (ODocument edge : edges) {
if (edge != null) {
otherVertex = edge.field(EDGE_FIELD_OUT);
if (otherVertex != null) {
otherEdges = otherVertex.field(VERTEX_FIELD_OUT);
*otherVertex.cloneOutToOriginalValue();*
if (otherEdges != null && otherEdges.remove(edge))
save(otherVertex);
}
delete(edge);
}
}
}
*// DELETE VERTEX AS DOCUMENT*
delete(iVertex);
commitBlock(safeMode);
} catch (RuntimeException e) {
rollbackBlock(safeMode);
throw e;
}
}
@SuppressWarnings("unchecked")
public void removeEdge(final ODocument iEdge) {
final boolean safeMode = beginBlock();
try {
final ODocument outVertex = iEdge.field(EDGE_FIELD_OUT);
if (outVertex != null) {
final Set<ODocument> out = ((Set<ODocument>)
outVertex.field(VERTEX_FIELD_OUT));
*outVertex.cloneOutToOriginalValue();*
if (out != null && out.remove(iEdge))
save(outVertex);
}
final ODocument inVertex = iEdge.field(EDGE_FIELD_IN);
if (inVertex != null) {
final Set<ODocument> in = ((Set<ODocument>)
inVertex.field(VERTEX_FIELD_IN));
* inVertex.cloneInToOriginalValue();*
if (in != null && in.remove(iEdge))
save(inVertex);
}
* // DELETE EDGE AS DOCUMENT*
delete(iEdge);
commitBlock(safeMode);
} catch (RuntimeException e) {
rollbackBlock(safeMode);
throw e;
}
}
==================================================================================
Note that when vertices are saved, they will be exactly the same as if
cloneInToOriginalValue or
cloneOutToOriginalValue were not called i.e. without any entries in the
_fieldOriginalValues.
We do not have to clear _fieldOriginalValues.
I'll be back – when we come up with a new non-hook solution ;-)
Best regards
Thomas
--

--

[Thomas]

Hi Luca,

Sorry that I haven't replied earlier but I just had to work on something
else for a short while but I'm back on track now.

I'll get back as soon as I know if the new features of the OMVRBTreeRIDSet
can fix the problem.

Best regards
Thomas

--

[Thomas]

Hi Luca,


I am sorry to say that the new features of the OMVRBTreeRIDSet do not fix
the problem.


I have worked on a solution to the history problem for several weeks now
and I have had

many meetings with my colleagues. We have been around several solution
models including

a solution using a database listener as a commit hook instead of a document
hook. Such a

solution has the advantage that we know how all vertices and edges which is
to be committed

are connected and we have thus eliminated the context problem described in
the previous

posts on this thread.


We would like to present our solution (when it is ready) but for now we
have some unresolved

issues related to clusters.


First: The history cluster solution has the fundamental flaw that pointers
to nodes in the

default space will be lost when moving the nodes to the history space. Thus
nodes in the

history space can contain dead pointers to deleted nodes in the default
space if nothing is

done. The problem arise when we delete a vertex or edge in the default
space and move it

to the history space. All nodes in the history space pointing to such a
deleted vertex or edge

will contain a "dead" pointer. A possible solution to this issue could be
to have something similar

to a pointer table in c, but containing ORID mappings of nodes that had
been moved to another

cluster, then corrections to all affected pointers (in other documents)
could be corrected on the fly.

Whenever we would retrieve a document containing a null pointer to another
document, we would

look up the ORID in the ORID table in order to find the new location in the
other cluster. We would

then update/clean up the pointers as they pop up and thus maintain all
references.


Second: We need to be able to search inside specific clusters using SQL.
Consider the following example.

We have four vertices (V1 to V4) and three edges (E1 to E3) as shown in the
figure below.



<Loading Image...>




















Assume that V1 has a field color = red that we will use in a SQL query. Now
we would

like to do two things here:


1: Find all vertices connected to V1 in the Default cluster (must ONLY
return V2).

select form cluster:Default where out.color='red'

This will return E1 so we try




select form cluster:Default where in.out.color='red'

This returns nothing even though V2 has E1 in "in" and V3 has E2 in "in"

Any suggestions?

What we would really like is something like:

select from cluster:Default V where connectedTo(V.color(red))


2: Find all vertices connected to V1 (must return V2, V3 and V4).

Would be nice with something like:

select from cluster:(Default, History) V where connectedTo(V.color(red))


Are we able to make such queries?


I have looked at the following wiki's and searched forum's for related
questions:


http://code.google.com/p/orient/wiki/GraphDatabase

http://code.google.com/p/orient/wiki/SQLWhere

http://code.google.com/p/orient/wiki/SQLQuery


Best regards

Post by Luca Garulli
Thomas

--

[Thomas]

Sorry for the disorder in my previous post and for posting a premature
question in
regards to searching graphs.

As for searching in different clusters and traversing a graph I am looking
at the

http://code.google.com/p/orient/wiki/SQLTraverse
http://code.google.com/p/orient/wiki/JavaTraverse

Maybe this can help me in the search example described before.


Best regards
Thomas

--

[Luca Garulli]

Post by Thomas
Hi Luca,
I am sorry to say that the new features of the OMVRBTreeRIDSet do not fix
the problem.

Since I've not your use case to try my changes was without a proper test
cases, but we could fix issue if we're pointed in the right direction.

... First: The history cluster solution has the fundamental flaw that

Post by Thomas
pointers to nodes...
...will contain a "dead" pointer. A possible solution to this issue could
be to have something similar
to a pointer table in c, but containing ORID mappings of nodes that had
been moved to another
cluster, then corrections to all affected pointers (in other documents)
could be corrected on the fly.
Whenever we would retrieve a document containing a null pointer to another
document, we would
look up the ORID in the ORID table in order to find the new location in
the other cluster. We would
then update/clean up the pointers as they pop up and thus maintain all
references.

This seems reasonable to me to have a consistent situation between real and
historical data. You could manage the pointer list in the trigger by
discovering all the links.

Post by Thomas
Second: We need to be able to search inside specific clusters using SQL.
Consider the following example.
We have four vertices (V1 to V4) and three edges (E1 to E3) as shown in
the figure below.
<https://lh3.googleusercontent.com/-hh3Pq_gulS8/UGVNfyaCkuI/AAAAAAAAAAM/m-FHDD4f-6E/s1600/SQL_Query_problem.png>
Assume that V1 has a field color = red that we will use in a SQL query.
Now we would
1: Find all vertices connected to V1 in the Default cluster (must ONLY
return V2).
select form cluster:Default where out.color='red'
This will return E1 so we try

Ok

Post by Thomas
select form cluster:Default where in.out.color='red'
This returns nothing even though V2 has E1 in "in" and V3 has E2 in "in"
Any suggestions?
select from cluster:Default V where connectedTo(V.color(red))

Your query is supposed to work.

Post by Thomas
2: Find all vertices connected to V1 (must return V2, V3 and V4).
select from cluster:(Default, History) V where connectedTo(V.color(red))

Why searching also in the history?

However once you've the RID of V1:

select from union( out.in, in.out ) from #10:45

or in general

select from union( out.in, in.out ) from V where in.out.color='red'
or out.in.color='red'

Are we able to make such queries?
Lvc@

--

[Luca Garulli]

Post by Thomas
2: Find all vertices connected to V1 (must return V2, V3 and V4).

Post by Thomas
select from cluster:(Default, History) V where connectedTo(V.color(red))

Why searching also in the history?
select from union( out.in, in.out ) from #10:45

This is better:

select flatten( union( out.in, in.out ) ) from #10:45

Post by Thomas
or in general
select from union( out.in, in.out ) from V where in.out.color='red'
or out.in.color='red'

This is better:

select flatten( union( out.in, in.out ) ) from cluster:default where
out.color = 'red' or in.color = 'red'

Lvc@

--

[Thomas]

This works (returns V2, V3 and V4)
select flatten( union( out.in, in.out ) ) from #10:45

This does not work
select flatten( union( out.in, in.out ) ) from cluster:default where
out.color = 'red' or in.color = 'red'

This does not work
select flatten( union( out.in, in.out ) ) from cluster:default where
in.out.color = 'red' or out.in.color = 'red'

This works (returns V2, V3 and V4)
select flatten( union( out.in, in.out ) ) from cluster:default where color
= 'red'

Getting all vertices connected to V1 is important when we need to work with
history, however the most important
query is the one where we get ONLY V2 i.e. in the default cluster V1 is
ONLY connected to V2. If we can't do
this there is no need for clusters at all i.e. if we have to filter
manually anyway then why use clusters.
The idea is to be able to query on all "active" nodes in the graph while
not having to consider the history
versions of the nodes.

Best regards
Thomas

--

[Luca Garulli]

Hi,
can you write here a query that returns V1 in any way, but by RID?

Post by Thomas
This works (returns V2, V3 and V4)
select flatten( union( out.in, in.out ) ) from #10:45
This does not work
select flatten( union( out.in, in.out ) ) from cluster:default where
out.color = 'red' or in.color = 'red'
This does not work
select flatten( union( out.in, in.out ) ) from cluster:default where
in.out.color = 'red' or out.in.color = 'red'
This works (returns V2, V3 and V4)
select flatten( union( out.in, in.out ) ) from cluster:default where
color = 'red'
Getting all vertices connected to V1 is important when we need to work
with history, however the most important
query is the one where we get ONLY V2 i.e. in the default cluster V1 is
ONLY connected to V2. If we can't do
this there is no need for clusters at all i.e. if we have to filter
manually anyway then why use clusters.
The idea is to be able to query on all "active" nodes in the graph while
not having to consider the history
versions of the nodes.
Best regards
Thomas
--

--

[Thomas]

Hi Luca,

We really want to avoid using RID's in our SQL queries. The criteria used
to find V1 is color = red

Thus there are two things we want to do here (in one query).

1. Select the vertex with color = red (should return V1 or the RID of V1)
2. Give us all vertices connected to V1 in default space (should return V2
or the RID of V2)

<Loading Image...>

--

[Thomas]

Hi Luca,

We really need your expert opinion/advise in this matter.

A fundamental and general problem with a graph history solution:

With time the number of edges will grow as more history edges and vertices
are added. If we assume that any history
version of a vertex will be added as a linked list then history versions of
the vertices will NOT be a problem (maybe a
disk size problem but not a search problem). However the history versions
of edges could potentially become a
problem because as we add new edges between vertices we never delete old
edges (we may move them to a
history space but they are not deleted).

NOW WE CAN SET UP A BASIC CRITERIA FOR A HISTORY SOLUTION:
All CRUD operations in a database with history must have very little
overhead compared to a database without
history. This basic criteria leads us to two fundamental questions:

1. How do we avoid that the growing number of history edges from each
vertex will eventually force the system to it's knees by making the CRUD
overhead intolerable.
2. Could a cluster solution effectively isolate non-history CRUD operations
from the history space or do we have to take into consideration that
non-history CRUD
operations will become slower over time. This last question relates to
the last two posts from us in this thread.

Best regards,
Thomas

--

[Luca Garulli]

Hi,
there are few rules to maintain an OrientDB fast even when grows up:

- few indexes
- work by RID as much as you can (in this way you avoid indexes for many
cases)
- as long as you can separate records in clusters

Now in your case the idea about linked list is very good because avoid
using the index for such task. If you always save the not-latest version of
vertices and edges in a historical cluster you avoid the overhead for
linear searches and you can always isolate the stuff to delete after a
while.

The problem is to maintain the consistency with:

- number of records: the user must see the effective number, not the sum
of the historical records
- indexes: you should always remove a historical record from the index!
This prevent to retrieve an old version of the graph element and leaves the
index not overloaded

These are generic suggestions, probably if you had something ready to be
published we can inspect the code to see better how we can improve ;-)

Post by Thomas
Hi Luca,
We really need your expert opinion/advise in this matter.
With time the number of edges will grow as more history edges and vertices
are added. If we assume that any history
version of a vertex will be added as a linked list then history versions
of the vertices will NOT be a problem (maybe a
disk size problem but not a search problem). However the history versions
of edges could potentially become a
problem because as we add new edges between vertices we never delete old
edges (we may move them to a
history space but they are not deleted).
All CRUD operations in a database with history must have very little
overhead compared to a database without
1. How do we avoid that the growing number of history edges from each
vertex will eventually force the system to it's knees by making the CRUD
overhead intolerable.
2. Could a cluster solution effectively isolate non-history CRUD
operations from the history space or do we have to take into consideration
that non-history CRUD
operations will become slower over time. This last question relates to
the last two posts from us in this thread.
Best regards,
Thomas
--

--

[Luca Garulli]

Hi,
probably there is something how you save vertices and edges in the default
cluster. Do you have a demo database where to play? Even on
www.nuvolabase.com ;-)

Post by Thomas
Hi Luca,
We really want to avoid using RID's in our SQL queries. The criteria used
to find V1 is color = red
Thus there are two things we want to do here (in one query).
1. Select the vertex with color = red (should return V1 or the RID of V1)
2. Give us all vertices connected to V1 in default space (should return V2
or the RID of V2)
<https://lh5.googleusercontent.com/-0MKsJ524CII/UGkxOjixFzI/AAAAAAAAAAk/f0RR0pthQtM/s1600/SQL_Query_problem.png>
--

--

[Shaohua Ma]

Hi,

I am new to the OrientDB. But the history concept is really cool. Is it
working yet? thanks.

Shaohua

Post by Thomas
Hi,
Using orientdb-graphed-1.0.1-SNAPSHOT I'm trying to create a history
solution using Hooks.
From inside the hooks (ONLY onRecordBeforeUpdate and onRecordBeforeDelete)
I want to do the
following which work except for the final save operation :-(
1. Copy a document that is about to be saved or deleted. The copy will be
used to create a history document.
2. Get all dirty fields from the document that is about to be updated and
then
3. Copy the original values from the ODocument (if any exist) and finally
4. save the document to a History cluster.
@Override
public boolean onRecordBeforeUpdate(ORecord<?> iRecord) {
if( iRecord instanceof ODocument ){
ODocument doc = (ODocument) iRecord;
if(validateClusterId(doc)) {
ODocument copy = doc.copy(); // Contains dirty fields that
we need to update with the old values before we can save the document to
History.
String[] dirtyFields = doc.getDirtyFields();
for(int i=0;i<dirtyFields.length;i++){
if(doc.getOriginalValue(dirtyFields[i]) == null)
System.out.println("Field [ " + dirtyFields[i] + " ] did not exist in
previous version");
else {
System.out.println("Original Value of dirty field
[ " + dirtyFields[i] +
" ] will be copied to History version.
Original value to be copied to the History version is [ " +
doc.getOriginalValue(dirtyFields[i]) + " ]
New value is [ " + doc.field(dirtyFields[i]) + " ]");
copy.field(dirtyFields[i],
doc.getOriginalValue(dirtyFields[i]));
}
}
copy.save("History");
}
}
return true;
}
Cannot update record #8:0 in storage 'orientdb' because the version is not
the latest.
Probably you are updating an old record or it has been modified by another
user (db=v1 your=v0)
I have also tried to use doc.getDatabase() and then create a new Vertex or
Edge, but this is not possible.
Any suggestions on how to save from inside the Hook?
Thanks

[Luca Garulli]

Hi Shaohua,
this isn't supported out of the box, but I don't know if any user did it.

Post by Shaohua Ma
Hi,
I am new to the OrientDB. But the history concept is really cool. Is it
working yet? thanks.
Shaohua

Hi,
Using orientdb-graphed-1.0.1-**SNAPSHOT I'm trying to create a history
solution using Hooks.
From inside the hooks (ONLY onRecordBeforeUpdate and
onRecordBeforeDelete) I want to do the
following which work except for the final save operation :-(
1. Copy a document that is about to be saved or deleted. The copy will be
used to create a history document.
2. Get all dirty fields from the document that is about to be updated and
then
3. Copy the original values from the ODocument (if any exist) and finally
4. save the document to a History cluster.
@Override
public boolean onRecordBeforeUpdate(ORecord<?**> iRecord) {
if( iRecord instanceof ODocument ){
ODocument doc = (ODocument) iRecord;
if(validateClusterId(doc)) {
ODocument copy = doc.copy(); // Contains dirty fields
that we need to update with the old values before we can save the document
to History.
String[] dirtyFields = doc.getDirtyFields();
for(int i=0;i<dirtyFields.length;i++){
if(doc.getOriginalValue(**dirtyFields[i]) == null)
System.out.println("Field [ " + dirtyFields[i] + " ] did not exist in
previous version");
else {
System.out.println("Original Value of dirty field
[ " + dirtyFields[i] +
** " ] will be copied to History version.
Original value to be copied to the History version is [ " +
** doc.getOriginalValue(**dirtyFields[i])
+ " ] New value is [ " + doc.field(dirtyFields[i]) + " ]");
copy.field(dirtyFields[i], doc.getOriginalValue(*
*dirtyFields[i]));
}
}
copy.save("History");
}
}
return true;
}
com.orientechnologies.orient.**core.exception.**
Cannot update record #8:0 in storage 'orientdb' because the version is
not the latest.
Probably you are updating an old record or it has been modified by
another user (db=v1 your=v0)
I have also tried to use doc.getDatabase() and then create a new Vertex
or Edge, but this is not possible.
Any suggestions on how to save from inside the Hook?
Thanks

--
---
You received this message because you are subscribed to the Google Groups "OrientDB" group.
To unsubscribe from this group and stop receiving emails from it, send an
For more options, visit https://groups.google.com/groups/opt_out.

[Asaf Shakarzy]

If you use Blueprints, then checkout this:
https://github.com/asaf/antiquity

Antiquity makes history support for any Blueprints graph.

Post by Luca Garulli
Hi Shaohua,
this isn't supported out of the box, but I don't know if any user did it.

Post by Shaohua Ma
Hi,
I am new to the OrientDB. But the history concept is really cool. Is it
working yet? thanks.
Shaohua

Post by Thomas
Hi,
Using orientdb-graphed-1.0.1-SNAPSHOT I'm trying to create a history
solution using Hooks.
From inside the hooks (ONLY onRecordBeforeUpdate and
onRecordBeforeDelete) I want to do the
following which work except for the final save operation :-(
1. Copy a document that is about to be saved or deleted. The copy will
be used to create a history document.
2. Get all dirty fields from the document that is about to be updated
and then
3. Copy the original values from the ODocument (if any exist) and finally
4. save the document to a History cluster.
@Override
public boolean onRecordBeforeUpdate(ORecord<?> iRecord) {
if( iRecord instanceof ODocument ){
ODocument doc = (ODocument) iRecord;
if(validateClusterId(doc)) {
ODocument copy = doc.copy(); // Contains dirty fields
that we need to update with the old values before we can save the document
to History.
String[] dirtyFields = doc.getDirtyFields();
for(int i=0;i<dirtyFields.length;i++){
if(doc.getOriginalValue(dirtyFields[i]) == null)
System.out.println("Field [ " + dirtyFields[i] + " ] did not exist in
previous version");
else {
System.out.println("Original Value of dirty
field [ " + dirtyFields[i] +
" ] will be copied to History version.
Original value to be copied to the History version is [ " +
doc.getOriginalValue(dirtyFields[i]) +
" ] New value is [ " + doc.field(dirtyFields[i]) + " ]");
copy.field(dirtyFields[i], doc.getOriginalValue(
dirtyFields[i]));
}
}
copy.save("History");
}
}
return true;
}
com.orientechnologies.orient.core.exception.
Cannot update record #8:0 in storage 'orientdb' because the version is
not the latest.
Probably you are updating an old record or it has been modified by
another user (db=v1 your=v0)
I have also tried to use doc.getDatabase() and then create a new Vertex
or Edge, but this is not possible.
Any suggestions on how to save from inside the Hook?
Thanks

--
---
You received this message because you are subscribed to the Google Groups
"OrientDB" group.
To unsubscribe from this group and stop receiving emails from it, send an
For more options, visit https://groups.google.com/groups/opt_out.

[Philipp Grosswiler]

Any progress on the time-based history implementation? It seems like some
years have passed since this discussion started and I wonder what are the
recommended ways on achieving this with OrientDB?

Post by Luca Garulli
Hi Shaohua,
this isn't supported out of the box, but I don't know if any user did it.

Post by Shaohua Ma
Hi,
I am new to the OrientDB. But the history concept is really cool. Is it
working yet? thanks.
Shaohua

Post by Thomas
Hi,
Using orientdb-graphed-1.0.1-SNAPSHOT I'm trying to create a history
solution using Hooks.
From inside the hooks (ONLY onRecordBeforeUpdate and
onRecordBeforeDelete) I want to do the
following which work except for the final save operation :-(
1. Copy a document that is about to be saved or deleted. The copy will
be used to create a history document.
2. Get all dirty fields from the document that is about to be updated
and then
3. Copy the original values from the ODocument (if any exist) and finally
4. save the document to a History cluster.
@Override
public boolean onRecordBeforeUpdate(ORecord<?> iRecord) {
if( iRecord instanceof ODocument ){
ODocument doc = (ODocument) iRecord;
if(validateClusterId(doc)) {
ODocument copy = doc.copy(); // Contains dirty fields
that we need to update with the old values before we can save the document
to History.
String[] dirtyFields = doc.getDirtyFields();
for(int i=0;i<dirtyFields.length;i++){
if(doc.getOriginalValue(dirtyFields[i]) == null)
System.out.println("Field [ " + dirtyFields[i] + " ] did not exist in
previous version");
else {
System.out.println("Original Value of dirty
field [ " + dirtyFields[i] +
" ] will be copied to History version.
Original value to be copied to the History version is [ " +
doc.getOriginalValue(dirtyFields[i]) +
" ] New value is [ " + doc.field(dirtyFields[i]) + " ]");
copy.field(dirtyFields[i], doc.getOriginalValue(
dirtyFields[i]));
}
}
copy.save("History");
}
}
return true;
}
com.orientechnologies.orient.core.exception.
Cannot update record #8:0 in storage 'orientdb' because the version is
not the latest.
Probably you are updating an old record or it has been modified by
another user (db=v1 your=v0)
I have also tried to use doc.getDatabase() and then create a new Vertex
or Edge, but this is not possible.
Any suggestions on how to save from inside the Hook?
Thanks

--
---
You received this message because you are subscribed to the Google Groups
"OrientDB" group.
To unsubscribe from this group and stop receiving emails from it, send an
For more options, visit https://groups.google.com/groups/opt_out.