From: Rusty Russell Date: Thu, 9 Sep 2010 07:25:43 +0000 (+0930) Subject: tdb2: update documentation. X-Git-Url: https://git.ozlabs.org/?p=ccan;a=commitdiff_plain;h=32710c917e41b6a283ab73190614623c1a8e9508 tdb2: update documentation. --- diff --git a/ccan/tdb2/doc/design.lyx b/ccan/tdb2/doc/design.lyx index 276832ea..8f061a7c 100644 --- a/ccan/tdb2/doc/design.lyx +++ b/ccan/tdb2/doc/design.lyx @@ -53,8 +53,8 @@ Rusty Russell, IBM Corporation \change_deleted 0 1283307542 26-July -\change_inserted 0 1283307544 -1-September +\change_inserted 0 1284016854 +9-September \change_unchanged -2010 \end_layout @@ -835,6 +835,18 @@ Internal locking is required to make sure that fcntl locks do not overlap \begin_layout Standard The aim is that building tdb with -DTDB_PTHREAD will result in a pthread-safe version of the library, and otherwise no overhead will exist. + +\change_inserted 0 1284016998 + Alternatively, a hooking mechanism similar to that proposed for +\begin_inset CommandInset ref +LatexCommand ref +reference "Proposed-Solution-locking-hook" + +\end_inset + + could be used to enable pthread locking at runtime. +\change_unchanged + \end_layout \begin_layout Subsection @@ -1183,6 +1195,109 @@ reference "TDB_CLEAR_IF_FIRST-Imposes-Performance" \end_inset . +\change_inserted 0 1284015637 + +\end_layout + +\begin_layout Subsection + +\change_inserted 0 1284015716 +Extending The Header Is Difficult +\end_layout + +\begin_layout Standard + +\change_inserted 0 1284015906 +We have reserved (zeroed) words in the TDB header, which can be used for + future features. + If the future features are compulsory, the version number must be updated + to prevent old code from accessing the database. + But if the future feature is optional, we have no way of telling if older + code is accessing the database or not. +\end_layout + +\begin_layout Subsubsection + +\change_inserted 0 1284015637 +Proposed Solution +\end_layout + +\begin_layout Standard + +\change_inserted 0 1284016114 +The header should contain a +\begin_inset Quotes eld +\end_inset + +format variant +\begin_inset Quotes erd +\end_inset + + value (64-bit). + This is divided into two 32-bit parts: +\end_layout + +\begin_layout Enumerate + +\change_inserted 0 1284016149 +The lower part reflects the format variant understood by code accessing + the database. +\end_layout + +\begin_layout Enumerate + +\change_inserted 0 1284016639 +The upper part reflects the format variant you must understand to write + to the database (otherwise you can only open for reading). +\end_layout + +\begin_layout Standard + +\change_inserted 0 1284016821 +The latter field can only be written at creation time, the former should + be written under the OPEN_LOCK when opening the database for writing, if + the variant of the code is lower than the current lowest variant. +\end_layout + +\begin_layout Standard + +\change_inserted 0 1284016803 +This should allow backwards-compatible features to be added, and detection + if older code (which doesn't understand the feature) writes to the database. +\change_deleted 0 1284016101 + +\end_layout + +\begin_layout Subsection + +\change_inserted 0 1284015634 +Record Headers Are Not Expandible +\end_layout + +\begin_layout Standard + +\change_inserted 0 1284015634 +If we later want to add (say) checksums on keys and data, it would require + another format change, which we'd like to avoid. +\end_layout + +\begin_layout Subsubsection + +\change_inserted 0 1284015634 +Proposed Solution +\end_layout + +\begin_layout Standard + +\change_inserted 0 1284016847 +We often have extra padding at the tail of a record. + If we ensure that the first byte (if any) of this padding is zero, we will + have a way for future changes to detect code which doesn't understand a + new format: the new code would write (say) a 1 at the tail, and thus if + there is no tail or the first byte is 0, we would know the extension is + not present on that record. +\change_unchanged + \end_layout \begin_layout Section diff --git a/ccan/tdb2/doc/design.lyx,v b/ccan/tdb2/doc/design.lyx,v index 47220267..54005d48 100644 --- a/ccan/tdb2/doc/design.lyx,v +++ b/ccan/tdb2/doc/design.lyx,v @@ -1,10 +1,15 @@ -head 1.8; +head 1.9; access; symbols; locks; strict; comment @# @; +1.9 +date 2010.09.09.07.25.12; author rusty; state Exp; +branches; +next 1.8; + 1.8 date 2010.09.02.02.29.05; author rusty; state Exp; branches; @@ -51,9 +56,9 @@ desc @ -1.8 +1.9 log -@Remove bogus footnote +@Extension mechanism. @ text @#LyX 1.6.5 created this file. For more info see http://www.lyx.org/ @@ -111,8 +116,8 @@ Rusty Russell, IBM Corporation \change_deleted 0 1283307542 26-July -\change_inserted 0 1283307544 -1-September +\change_inserted 0 1284016854 +9-September \change_unchanged -2010 \end_layout @@ -893,6 +898,18 @@ Internal locking is required to make sure that fcntl locks do not overlap \begin_layout Standard The aim is that building tdb with -DTDB_PTHREAD will result in a pthread-safe version of the library, and otherwise no overhead will exist. + +\change_inserted 0 1284016998 + Alternatively, a hooking mechanism similar to that proposed for +\begin_inset CommandInset ref +LatexCommand ref +reference "Proposed-Solution-locking-hook" + +\end_inset + + could be used to enable pthread locking at runtime. +\change_unchanged + \end_layout \begin_layout Subsection @@ -1241,6 +1258,109 @@ reference "TDB_CLEAR_IF_FIRST-Imposes-Performance" \end_inset . +\change_inserted 0 1284015637 + +\end_layout + +\begin_layout Subsection + +\change_inserted 0 1284015716 +Extending The Header Is Difficult +\end_layout + +\begin_layout Standard + +\change_inserted 0 1284015906 +We have reserved (zeroed) words in the TDB header, which can be used for + future features. + If the future features are compulsory, the version number must be updated + to prevent old code from accessing the database. + But if the future feature is optional, we have no way of telling if older + code is accessing the database or not. +\end_layout + +\begin_layout Subsubsection + +\change_inserted 0 1284015637 +Proposed Solution +\end_layout + +\begin_layout Standard + +\change_inserted 0 1284016114 +The header should contain a +\begin_inset Quotes eld +\end_inset + +format variant +\begin_inset Quotes erd +\end_inset + + value (64-bit). + This is divided into two 32-bit parts: +\end_layout + +\begin_layout Enumerate + +\change_inserted 0 1284016149 +The lower part reflects the format variant understood by code accessing + the database. +\end_layout + +\begin_layout Enumerate + +\change_inserted 0 1284016639 +The upper part reflects the format variant you must understand to write + to the database (otherwise you can only open for reading). +\end_layout + +\begin_layout Standard + +\change_inserted 0 1284016821 +The latter field can only be written at creation time, the former should + be written under the OPEN_LOCK when opening the database for writing, if + the variant of the code is lower than the current lowest variant. +\end_layout + +\begin_layout Standard + +\change_inserted 0 1284016803 +This should allow backwards-compatible features to be added, and detection + if older code (which doesn't understand the feature) writes to the database. +\change_deleted 0 1284016101 + +\end_layout + +\begin_layout Subsection + +\change_inserted 0 1284015634 +Record Headers Are Not Expandible +\end_layout + +\begin_layout Standard + +\change_inserted 0 1284015634 +If we later want to add (say) checksums on keys and data, it would require + another format change, which we'd like to avoid. +\end_layout + +\begin_layout Subsubsection + +\change_inserted 0 1284015634 +Proposed Solution +\end_layout + +\begin_layout Standard + +\change_inserted 0 1284016847 +We often have extra padding at the tail of a record. + If we ensure that the first byte (if any) of this padding is zero, we will + have a way for future changes to detect code which doesn't understand a + new format: the new code would write (say) a 1 at the tail, and thus if + there is no tail or the first byte is 0, we would know the extension is + not present on that record. +\change_unchanged + \end_layout \begin_layout Section @@ -2519,6 +2639,20 @@ At some later point, a sync would allow recovery of the old data into the @ +1.8 +log +@Remove bogus footnote +@ +text +@d56 2 +a57 2 +\change_inserted 0 1283307544 +1-September +d838 12 +d1198 103 +@ + + 1.7 log @Moving hash table does not work. diff --git a/ccan/tdb2/doc/design.pdf b/ccan/tdb2/doc/design.pdf index 87242edc..52d0f9b0 100644 Binary files a/ccan/tdb2/doc/design.pdf and b/ccan/tdb2/doc/design.pdf differ diff --git a/ccan/tdb2/doc/design.txt b/ccan/tdb2/doc/design.txt index 88334a8a..967c0b09 100644 --- a/ccan/tdb2/doc/design.txt +++ b/ccan/tdb2/doc/design.txt @@ -2,7 +2,7 @@ TDB2: A Redesigning The Trivial DataBase Rusty Russell, IBM Corporation -1-September-2010 +9-September-2010 Abstract @@ -277,7 +277,9 @@ maintained. The aim is that building tdb with -DTDB_PTHREAD will result in a pthread-safe version of the library, and otherwise no overhead -will exist. +will exist. Alternatively, a hooking mechanism similar to that +proposed for [Proposed-Solution-locking-hook] could be used to +enable pthread locking at runtime. 2.8 *_nonblock Functions And *_mark Functions Expose Implementation @@ -473,6 +475,50 @@ it alone has opened the TDB and will erase it. Remove TDB_CLEAR_IF_FIRST. Other workarounds are possible, but see [TDB_CLEAR_IF_FIRST-Imposes-Performance]. +2.15 Extending The Header Is Difficult + +We have reserved (zeroed) words in the TDB header, which can be +used for future features. If the future features are compulsory, +the version number must be updated to prevent old code from +accessing the database. But if the future feature is optional, we +have no way of telling if older code is accessing the database or +not. + +2.15.1 Proposed Solution + +The header should contain a “format variant” value (64-bit). This +is divided into two 32-bit parts: + +1. The lower part reflects the format variant understood by code + accessing the database. + +2. The upper part reflects the format variant you must understand + to write to the database (otherwise you can only open for + reading). + +The latter field can only be written at creation time, the former +should be written under the OPEN_LOCK when opening the database +for writing, if the variant of the code is lower than the current +lowest variant. + +This should allow backwards-compatible features to be added, and +detection if older code (which doesn't understand the feature) +writes to the database. + +2.16 Record Headers Are Not Expandible + +If we later want to add (say) checksums on keys and data, it +would require another format change, which we'd like to avoid. + +2.16.1 Proposed Solution + +We often have extra padding at the tail of a record. If we ensure +that the first byte (if any) of this padding is zero, we will +have a way for future changes to detect code which doesn't +understand a new format: the new code would write (say) a 1 at +the tail, and thus if there is no tail or the first byte is 0, we +would know the extension is not present on that record. + 3 Performance And Scalability Issues 3.1 TDB_CLEAR_IF_FIRST