Request 813 (accepted)

We truncated the diff of some files because they were too big. If you want to see the full diff for every file, click here.

You're not reviewing the full diff of request 813 , but the diff to the superseded request 782 (Show full diff)

Overview

Request 813 (accepted)

Applied patch to fix ctl_mboxlist segfault

Created by dhoffend almost 10 years ago
In state accepted
Supersedes 782

Submit cyrus-imapd

Submit package home:dhoffend...b:3.3:Updates / cyrus-imapd to package Kolab:3.3:Updates / cyrus-imapd

cyrus-imapd.spec Changed

@@ -1,4 +1,4 @@
-%if 0%{?opensuse_bs} && 0%{?suse_version} < 1
+%if 0%{?opensuse_bs}
 #!BuildIgnore:  systemd
 %endif
 
@@ -38,13 +38,13 @@
 Name:               cyrus-imapd
 Summary:            A high-performance mail server with IMAP, POP3, NNTP and SIEVE support
 Version:            2.5
-Release:            0.2.dev20141209.gitf0131008%{?dist}
+Release:            0.1.dev20140813.git3d45146a%{?dist}
 License:            BSD
 Group:              System Environment/Daemons
 URL:                http://www.cyrusimap.org
 
 # Upstream sources
-# From f01310085cbf58958412deda9c61ecfd7c82a19b
+# From 3d45146a7befa1651d46a4db381fb10be40e53ee
 Source0:            ftp://ftp.andrew.cmu.edu/pub/cyrus/%{_name}-%{real_version}%{?dot_snapshot_version}.tar.gz
 Source1:            cyrus-imapd.imap-2.3.x-conf
 Source2:            cyrus-imapd.cvt_cyrusdb_all
@@ -67,8 +67,8 @@
 ## Patches
 ##
 
-Patch0001:          cyrus-imapd-2.5-ctl_mboxlist-mbtype.patch
-Patch0002:          cyrus-imapd-2.5-revert-safe-skiplist.patch
+Patch0001:          cyrus-imapd-2.5-default-twoskip.patch
+Patch0002:          cyrus-imapd-2.5-ctl_mboxlist-mbtype.patch
 
 BuildRoot:          %(mktemp -ud %{_tmppath}/%{name}-%{version}-%{release}-XXXXXX)
 
@@ -79,14 +79,6 @@
 BuildRequires:      automake
 BuildRequires:      bison
 BuildRequires:      cyrus-sasl-devel
-%if 0%{?suse_version}
-BuildRequires:      cunit-devel
-BuildRequires:      cyrus-sasl-digestmd5
-%else
-BuildRequires:      CUnit-devel
-BuildRequires:      cyrus-sasl-md5
-%endif
-BuildRequires:      cyrus-sasl-plain
 
 %if 0%{?with_bdb}
 %if 0%{?suse_version}
@@ -120,7 +112,10 @@
 BuildRequires:      libical-devel
 %endif
 
+%if 0%{?suse_version} || 0%{?rhel} > 5
 BuildRequires:      libtool
+%endif
+
 BuildRequires:      libuuid-devel
 
 %if 0%{?with_mysql}
@@ -153,12 +148,6 @@
 BuildRequires:      sqlite-devel
 %endif
 
-%if 0%{?with_systemd}
-%if 0%{?suse_version}
-BuildRequires:      systemd
-%endif
-%endif
-
 %if 0%{?with_tcpwrap}
 %if 0%{?suse_version}
 BuildRequires:      tcpd-devel
@@ -199,7 +188,9 @@
 %if 0%{?with_systemd}
 %if 0%{?suse_version}
 PreReq:             %fillup_prereq
-%{?systemd_requires}
+Requires(post):     systemd
+Requires(preun):    systemd
+Requires(postun):   systemd
 %else
 %if 0%{?opensuse_bs} == 0
 Requires(post):     systemd-units
@@ -264,14 +255,9 @@
 %prep
 %setup -q -n %{_name}-%{real_version}%{?dot_snapshot_version}
 
-%patch0001 -p1
+#%patch0001 -p1
 %patch0002 -p1
 
-%if 0%{?with_bdb} < 1
-sed -i -e 's/,berkeley//g' cunit/db.testc
-sed -r -i -e 's/"berkeley(|-a-z-+)", //g' lib/imapoptions
-%endif
-
 # only to update config.* files
 aclocal -I cmulocal
 libtoolize --install
@@ -299,6 +285,7 @@
 # enable idled in .conf files to prevent error messages
 %{__perl} -pi -e "s/#  idled/  idled/" master/conf/*.conf
 
+
 %build
 CPPFLAGS="%{?optflags} -I%{_includedir}/et -I%{_includedir}/kerberosIV"; export CPPFLAGS
 CFLAGS="%{?optflags} -fPIC"; export CFLAGS
@@ -316,7 +303,6 @@
     --enable-netscapehack \
     --enable-nntp \
     --enable-replication \
-    --enable-unit-tests \
 %if 0
     --enable-xapian \
 %endif
@@ -375,6 +361,7 @@
   %{buildroot}%{_var}/spool/imap \
   %{buildroot}%{_var}/lib/imap/{user,quota,proc,log,msg,socket,db,sieve,sync,md5,rpm,backup,meta} \
   %{buildroot}%{_var}/lib/imap/ptclient \
+  %{buildroot}%{_var}/lib/imap/rpm \
   %{buildroot}%{_sysconfdir}/pki/%{_name} \
   doc/contrib
 
@@ -428,7 +415,7 @@
 %{__install} -m 644 master/conf/*.conf doc/conf/
 
 # Generate db config file
-( grep '^{' lib/imapoptions | grep _db | grep -v _db_path | cut -d'"' -f 2,4 | \
+( grep '^{' lib/imapoptions | grep _db | cut -d'"' -f 2,4 | \
   sed -e 's/^ *//' -e 's/-nosync//' -e 's/ *$//' -e 's/"/=/'
   echo sieve_version=2.2.3 ) | sort > %{buildroot}%{_var}/lib/imap/rpm/db.cfg
 
@@ -471,9 +458,6 @@
 %{__rm} -f %{buildroot}%{_mandir}/man8/syncnews.8*
 find %{buildroot}%{perl_vendorarch} -type f -name "*.bs" -exec %{__rm} -vf {} \;
 
-%check
-make check || :
-
 %clean
 %{__rm} -rf %{buildroot}
 
@@ -508,7 +492,13 @@
     source %{_sysconfdir}/sysconfig/cyrus-imapd
 fi
 
-if  $CHATTRSYNC -eq 1 ; then
+if  $CHATTRSYNC -eq 0 ; then
+    # Force synchronous updates of files in the following directories
+    chattr -R -S \
+        $(grep ^configdirectory: /etc/imapd.conf | cut -d':' -f2) \
+        $(grep ^partition- /etc/imapd.conf | cut -d':' -f2) \
+        $(grep ^metapartition- /etc/imapd.conf | cut -d':' -f2) 2>/dev/null ||:
+else
     # Force synchronous updates of files in the following directories
     chattr -R +S \
         $(grep ^configdirectory: /etc/imapd.conf | cut -d':' -f2) \
@@ -557,7 +547,7 @@
 %if 0%{?suse_version}
     %service_del_preun cyrus-imapd.service
 %else
-    %systemd_preun cyrus-imapd.service >/dev/null 2>&1 || :
+    %systemd_preun cyrus-imapd.service
 %endif
 %else
     if  $1 = 0 ; then
@@ -600,7 +590,7 @@
 %if 0%{?suse_version}
 %config(noreplace) %{_localstatedir}/adm/fillup-templates/sysconfig.%{_name}
 %else
-%attr(0640,root,%{_cyrusgroup}) %config(noreplace) %{_sysconfdir}/sysconfig/%{_name}
+%config(noreplace) %{_sysconfdir}/sysconfig/%{_name}
 %endif
 %config(noreplace) %{_sysconfdir}/pam.d/pop
 %config(noreplace) %{_sysconfdir}/pam.d/imap
@@ -613,7 +603,7 @@
 %{_bindir}/imtest
 %{_bindir}/installsieve
 %{_bindir}/lmtptest
-%{_bindir}/mupdatetest
+%{_bindir}/mupatetest
 %{_bindir}/nntptest
 %{_bindir}/pop3test
 %{_bindir}/sieveshell
@@ -750,16 +740,6 @@
 %{_libdir}/*.la
 
 %changelog
-* Tue Dec  9 2014 Jeroen van Meeuwen <vanmeeuwen@kolabsys.com> - 2.5-11.git
-- Fix Murder topologies not handling null ACLs at all
-
-* Thu Sep 25 2014 Jeroen van Meeuwen <vanmeeuwen@kolabsys.com> - 2.5-10.git
-- Fix event notifications
-
-* Fri Aug 29 2014 Jeroen van Meeuwen <vanmeeuwen@kolabsys.com> - 2.5-9.git
-- Snapshot of (b2ef80be)
-- Merge enhanced Sieve Date and Index extension
-
 * Mon Mar  3 2014 Jeroen van Meeuwen <vanmeeuwen@kolabsys.com> - 2.5-8.git
 - Fix shared folder deleted namespace prefix

cyrus-imapd-2.5-default-twoskip.patch Added

@@ -0,0 +1,80 @@
+diff --git a/lib/imapoptions b/lib/imapoptions
+index 2c477a6..78c5f46 100644
+--- a/lib/imapoptions
++++ b/lib/imapoptions
+@@ -163,7 +163,7 @@ are listed with ``<none>''.
+    affect LMTP delivery of messages directly to mailboxes via
+    plus-addressing. */
+ 
+-{ "annotation_db", "skiplist", STRINGLIST("berkeley", "berkeley-hash", "skiplist", "twoskip")}
++{ "annotation_db", "twoskip", STRINGLIST("berkeley", "berkeley-hash", "skiplist", "twoskip")}
+ /* The cyrusdb backend to use for mailbox annotations. */
+ 
+ { "annotation_db_path", NULL, STRING }
+@@ -413,7 +413,7 @@ Blank lines and lines beginning with ``#'' are ignored.
+    session.  Otherwise, the missing mailbox is treated as empty while
+    in use by the client.*/
+ 
+-{ "duplicate_db", "skiplist", STRINGLIST("berkeley", "berkeley-nosync", "berkeley-hash", "berkeley-hash-nosync", "skiplist", "sql", "twoskip")}
++{ "duplicate_db", "twoskip", STRINGLIST("berkeley", "berkeley-nosync", "berkeley-hash", "berkeley-hash-nosync", "skiplist", "sql", "twoskip")}
+ /* The cyrusdb backend to use for the duplicate delivery suppression
+    and sieve. */
+ 
+@@ -841,10 +841,10 @@ Blank lines and lines beginning with ``#'' are ignored.
+ { "maxword", 131072, INT }
+ /* Maximum size of a single word for the parser.  Default 128k */
+ 
+-{ "mboxkey_db", "skiplist", STRINGLIST("berkeley", "skiplist", "twoskip") }
++{ "mboxkey_db", "twoskip", STRINGLIST("berkeley", "skiplist", "twoskip") }
+ /* The cyrusdb backend to use for mailbox keys. */
+ 
+-{ "mboxlist_db", "skiplist", STRINGLIST("flat", "berkeley", "berkeley-hash", "skiplist", "sql", "twoskip")}
++{ "mboxlist_db", "twoskip", STRINGLIST("flat", "berkeley", "berkeley-hash", "skiplist", "sql", "twoskip")}
+ /* The cyrusdb backend to use for the mailbox list. */
+ 
+ { "mboxlist_db_path", NULL, STRING }
+@@ -1163,7 +1163,7 @@ And the notification message will be available on \fIstdin\fR.
+ /* Unix domain socket that ptloader listens on.
+    (defaults to configdir/ptclient/ptsock) */
+ 
+-{ "ptscache_db", "skiplist", STRINGLIST("berkeley", "berkeley-hash", "skiplist", "twoskip")}
++{ "ptscache_db", "twoskip", STRINGLIST("berkeley", "berkeley-hash", "skiplist", "twoskip")}
+ /* The cyrusdb backend to use for the pts cache. */
+ 
+ { "ptscache_db_path", NULL, STRING }
+@@ -1188,7 +1188,7 @@ And the notification message will be available on \fIstdin\fR.
+ /* This specifies the Class Selector or Differentiated Services Code Point
+    designation on IP headers (in the ToS field). */
+ 
+-{ "quota_db", "quotalegacy", STRINGLIST("flat", "berkeley", "berkeley-hash", "skiplist", "sql", "quotalegacy", "twoskip")}
++{ "quota_db", "twoskip", STRINGLIST("flat", "berkeley", "berkeley-hash", "skiplist", "sql", "quotalegacy", "twoskip")}
+ /* The cyrusdb backend to use for quotas. */
+ 
+ { "quota_db_path", NULL, STRING }
+@@ -1269,7 +1269,7 @@ And the notification message will be available on \fIstdin\fR.
+    recommended for most cases - it's a good compromise which
+    keeps words separate. */
+ 
+-{ "seenstate_db", "skiplist", STRINGLIST("flat", "berkeley", "berkeley-hash", "skiplist", "twoskip")}
++{ "seenstate_db", "twoskip", STRINGLIST("flat", "berkeley", "berkeley-hash", "skiplist", "twoskip")}
+ /* The cyrusdb backend to use for the seen state. */
+ 
+ { "sendmail", "/usr/lib/sendmail", STRING }
+@@ -1440,7 +1440,7 @@ product version in the capabilities */
+ { "statuscache", 0, SWITCH }
+ /* Enable/disable the imap status cache. */
+ 
+-{ "statuscache_db", "skiplist", STRINGLIST("berkeley", "berkeley-nosync", "berkeley-hash", "berkeley-hash-nosync", "skiplist", "sql", "twoskip") }
++{ "statuscache_db", "twoskip", STRINGLIST("berkeley", "berkeley-nosync", "berkeley-hash", "berkeley-hash-nosync", "skiplist", "sql", "twoskip") }
+ /* The cyrusdb backend to use for the imap status cache. */
+ 
+ { "statuscache_db_path", NULL, STRING }
+@@ -1545,7 +1545,7 @@ product version in the capabilities */
+    have filenames with the hashed value of the certificates (see
+    openssl(XXX)). */
+ 
+-{ "tlscache_db", "skiplist", STRINGLIST("berkeley", "berkeley-nosync", "berkeley-hash", "berkeley-hash-nosync", "skiplist", "sql", "twoskip")}
++{ "tlscache_db", "twoskip", STRINGLIST("berkeley", "berkeley-nosync", "berkeley-hash", "berkeley-hash-nosync", "skiplist", "sql", "twoskip")}
+ /* The cyrusdb backend to use for the TLS cache. */
+ 
+ { "tlscache_db_path", NULL, STRING }

cyrus-imapd-2.5-openssl-stack.patch Deleted

cyrus-imapd-2.5-revert-safe-skiplist.patch Deleted

@@ -1,520 +0,0 @@
-diff --git a/lib/cyrusdb_skiplist.c b/lib/cyrusdb_skiplist.c
-index 18d3ca3..fc540e0 100644
---- a/lib/cyrusdb_skiplist.c
-+++ b/lib/cyrusdb_skiplist.c
-@@ -399,7 +399,7 @@ static unsigned RECSIZE_safe(struct dbengine *db, const char *ptr)
-     case ADD:
- 	level = LEVEL_safe(db, ptr);
- 	if (!level) {
--	    syslog(LOG_ERR, "IOERROR: skiplist RECSIZE not safe %s, offset %u",
-+	    syslog(LOG_ERR, "IOERROR: skiplist2 RECSIZE_safe not safe %s, offset %u",
- 		   db->fname, (unsigned)(ptr - db->map_base));
- 	    return 0;
- 	}
-@@ -413,20 +413,50 @@ static unsigned RECSIZE_safe(struct dbengine *db, const char *ptr)
- 	break;
- 
-     case DELETE:
--	if (!is_safe(db, ptr+8)) {
--	    syslog(LOG_ERR, "IOERROR: skiplist RECSIZE not safe %s, offset %u",
--		   db->fname, (unsigned)(ptr - db->map_base));
--	    return 0;
--	}
- 	ret += 8;
- 	break;
- 
-     case COMMIT:
--	if (!is_safe(db, ptr+4)) {
--	    syslog(LOG_ERR, "IOERROR: skiplist RECSIZE not safe %s, offset %u",
--		   db->fname, (unsigned)(ptr - db->map_base));
--	    return 0;
--	}
-+	ret += 4;
-+	break;
-+    }
-+
-+    return ret;
-+}
-+
-+/* how many levels does this record have? */
-+static unsigned LEVEL(const char *ptr)
-+{
-+    const uint32_t *p, *q;
-+
-+    assert(TYPE(ptr) == DUMMY || TYPE(ptr) == INORDER || TYPE(ptr) == ADD);
-+    p = q = (uint32_t *) FIRSTPTR(ptr);
-+    while (*p != (uint32_t)-1) p++;
-+    return (p - q);
-+}
-+
-+/* how big is this record? */
-+static unsigned RECSIZE(const char *ptr)
-+{
-+    int ret = 0;
-+    switch (TYPE(ptr)) {
-+    case DUMMY:
-+    case INORDER:
-+    case ADD:
-+	ret += 4;			/* tag */
-+	ret += 4;			/* keylen */
-+	ret += ROUNDUP(KEYLEN(ptr));    /* key */
-+	ret += 4;			/* datalen */
-+	ret += ROUNDUP(DATALEN(ptr));   /* data */
-+	ret += 4 * LEVEL(ptr);	        /* pointers */
-+	ret += 4;			/* padding */
-+	break;
-+
-+    case DELETE:
-+	ret += 8;
-+	break;
-+
-+    case COMMIT:
- 	ret += 4;
- 	break;
-     }
-@@ -494,12 +524,7 @@ static int newtxn(struct dbengine *db, struct txn **tidptr)
- }
- 
- 
--static unsigned PADDING_safe(struct dbengine *db, const char *ptr)
--{
--    unsigned size = RECSIZE_safe(db, ptr);
--    if (!size) return 0;
--    return ntohl(*((uint32_t *)((ptr) + size - 4)));
--}
-+#define PADDING(ptr) (ntohl(*((uint32_t *)((ptr) + RECSIZE(ptr) - 4))))
- 
- /* given an open, mapped db, read in the header information */
- static int read_header(struct dbengine *db)
-@@ -570,9 +595,9 @@ static int read_header(struct dbengine *db)
- 	       db->fname);
- 	r = CYRUSDB_IOERROR;
-     }
--    if (!r && LEVEL_safe(db, dptr) != db->maxlevel) {
-+    if (!r && LEVEL(dptr) != db->maxlevel) {
- 	syslog(LOG_ERR, "DBERROR: %s: DUMMY level(%d) != db->maxlevel(%d)",
--	       db->fname, LEVEL_safe(db, dptr), db->maxlevel);
-+	       db->fname, LEVEL(dptr), db->maxlevel);
- 	r = CYRUSDB_IOERROR;
-     }
- 
-@@ -1281,7 +1306,7 @@ static int mystore(struct dbengine *db,
- 	    return CYRUSDB_EXISTS;
- 	} else {
- 	    /* replace with an equal height node */
--	    lvl = LEVEL_safe(db, ptr);
-+	    lvl = LEVEL(ptr);
- 
- 	    /* log a removal */
- 	    WRITEV_ADD_TO_IOVEC(iov, num_iov, (char *) &delrectype, 4);
-@@ -1575,8 +1600,8 @@ static int myabort(struct dbengine *db, struct txn *tid)
- 
- 	/* find the last log entry */
- 	for (offset = tid->logstart, ptr = db->map_base + offset; 
--	     offset + RECSIZE_safe(db, ptr) != (uint32_t) tid->logend;
--	     offset += RECSIZE_safe(db, ptr), ptr = db->map_base + offset) ;
-+	     offset + RECSIZE(ptr) != (uint32_t) tid->logend;
-+	     offset += RECSIZE(ptr), ptr = db->map_base + offset) ;
- 	
- 	offset = ptr - db->map_base;
- 
-@@ -1613,7 +1638,7 @@ static int myabort(struct dbengine *db, struct txn *tid)
- 	    /* re-add this record.  it can't exist right now. */
- 	    netnewoffset = *((uint32_t *)(ptr + 4));
- 	    q = db->map_base + ntohl(netnewoffset);
--	    lvl = LEVEL_safe(db, q);
-+	    lvl = LEVEL(q);
- 	    (void) find_node(db, KEY(q), KEYLEN(q), updateoffsets);
- 	    for (i = 0; i < lvl; i++) {
- 		/* the current pointers FROM this node are correct,
-@@ -1628,7 +1653,7 @@ static int myabort(struct dbengine *db, struct txn *tid)
- 	}
- 
- 	/* remove looking at this */
--	tid->logend -= RECSIZE_safe(db, ptr);
-+	tid->logend -= RECSIZE(ptr);
-     }
- 
-     /* truncate the file to remove log entries */
-@@ -1742,13 +1767,13 @@ static int mycheckpoint(struct dbengine *db)
- 	uint32_t netnewoffset;
- 
- 	ptr = db->map_base + offset;
--	lvl = LEVEL_safe(db, ptr);
-+	lvl = LEVEL(ptr);
- 	db->listsize++;
- 
- 	num_iov = 0;
- 	WRITEV_ADD_TO_IOVEC(iov, num_iov, (char *) &iorectype, 4);
- 	/* copy all but the rectype from the record */
--	WRITEV_ADD_TO_IOVEC(iov, num_iov, (char *) ptr + 4, RECSIZE_safe(db, ptr) - 4);
-+	WRITEV_ADD_TO_IOVEC(iov, num_iov, (char *) ptr + 4, RECSIZE(ptr) - 4);
- 
- 	newoffset = lseek(db->fd, 0, SEEK_END);
- 	netnewoffset = htonl(newoffset);
-@@ -1915,9 +1940,9 @@ static int dump(struct dbengine *db, int detail __attribute__((unused)))
- 	case INORDER:
- 	case ADD:
- 	    printf("kl=%d dl=%d lvl=%d\n",
--		   KEYLEN(ptr), DATALEN(ptr), LEVEL_safe(db, ptr));
-+		   KEYLEN(ptr), DATALEN(ptr), LEVEL(ptr));
- 	    printf("\t");
--	    for (i = 0; i < LEVEL_safe(db, ptr); i++) {
-+	    for (i = 0; i < LEVEL(ptr); i++) {
- 		printf("%04X ", FORWARD(ptr, i));
- 	    }
- 	    printf("\n");
-@@ -1932,7 +1957,7 @@ static int dump(struct dbengine *db, int detail __attribute__((unused)))
- 	    break;
- 	}
- 
--	ptr += RECSIZE_safe(db, ptr);
-+	ptr += RECSIZE(ptr);
-     }
- 
-     unlock(db);
-@@ -1961,7 +1986,7 @@ static int myconsistent(struct dbengine *db, struct txn *tid, int locked)
- 
- 	ptr = db->map_base + offset;
- 
--	for (i = 0; i < LEVEL_safe(db, ptr); i++) {
-+	for (i = 0; i < LEVEL(ptr); i++) {
- 	    offset = FORWARD(ptr, i);
- 
- 	    if (offset > db->map_size) {
-@@ -2005,11 +2030,9 @@ static int myconsistent(struct dbengine *db, struct txn *tid, int locked)
- static int recovery(struct dbengine *db, int flags)
- {
-     const char *ptr, *keyptr;
--    unsigned filesize = db->map_size;
-     unsigned updateoffsetsSKIPLIST_MAXLEVEL+1;
-     uint32_t offset, offsetnet, myoff = 0;
--    int r = 0;
--    int need_checkpoint = libcyrus_config_getswitch(CYRUSOPT_SKIPLIST_ALWAYS_CHECKPOINT);
-+    int r = 0, need_checkpoint = 0;
-     time_t start = time(NULL);
-     unsigned i;
- 
-@@ -2063,11 +2086,11 @@ static int recovery(struct dbengine *db, int flags)
-     }
- 
-     /* pointers for db->maxlevel */
--    if (!r && LEVEL_safe(db, ptr) != db->maxlevel) {
-+    if (!r && LEVEL(ptr) != db->maxlevel) {
- 	r = CYRUSDB_IOERROR;
- 	syslog(LOG_ERR, 
- 	       "DBERROR: skiplist recovery %s: dummy node level: %d != %d",
--	       db->fname, LEVEL_safe(db, ptr), db->maxlevel);
-+	       db->fname, LEVEL(ptr), db->maxlevel);
-     }
-     
-     for (i = 0; i < db->maxlevel; i++) {
-@@ -2078,7 +2101,7 @@ static int recovery(struct dbengine *db, int flags)
-     
-     /* reset the data that was written INORDER by the last checkpoint */
-     offset = DUMMY_OFFSET(db) + DUMMY_SIZE(db);
--    while (!r && (offset < filesize)
-+    while (!r && (offset < db->map_size)
- 	      && TYPE(db->map_base + offset) == INORDER) {
- 	ptr = db->map_base + offset;
- 	offsetnet = htonl(offset);
-@@ -2088,9 +2111,9 @@ static int recovery(struct dbengine *db, int flags)
- 	/* xxx check \0 fill on key */
- 
- 	/* xxx check \0 fill on data */
--
-+	    
- 	/* update previous pointers, record these for updating */
--	for (i = 0; !r && i < LEVEL_safe(db, ptr); i++) {
-+	for (i = 0; !r && i < LEVEL(ptr); i++) {
- 	    r = lseek(db->fd, updateoffsetsi, SEEK_SET);
- 	    if (r < 0) {
- 		syslog(LOG_ERR, "DBERROR: lseek %s: %m", db->fname);
-@@ -2113,23 +2136,15 @@ static int recovery(struct dbengine *db, int flags)
- 	    updateoffsetsi = offset + (PTR(ptr, i) - ptr);
- 	}
- 
--	if (!r) {
--	    unsigned size = RECSIZE_safe(db, ptr);
--	    if (!size) {
--		syslog(LOG_ERR, "skiplist recovery %s: damaged record at %u, truncating here",
--		       db->fname, offset);
--		filesize = offset;
--		break;
--	    }
--
--	    if (PADDING_safe(db, ptr) != (uint32_t) -1) {
--		syslog(LOG_ERR, "DBERROR: %s: offset %04X padding not -1",
--		       db->fname, offset);
--		filesize = offset;
--		break;
--	    }
-+	/* check padding */
-+	if (!r && PADDING(ptr) != (uint32_t) -1) {
-+	    syslog(LOG_ERR, "DBERROR: %s: offset %04X padding not -1",
-+		   db->fname, offset);
-+	    r = CYRUSDB_IOERROR;
-+	}
- 
--	    offset += size;
-+	if (!r) {
-+	    offset += RECSIZE(ptr);
- 	}
-     }
- 
-@@ -2164,7 +2179,7 @@ static int recovery(struct dbengine *db, int flags)
-     }
- 
-     /* replay the log */
--    while (!r && offset < filesize) {
-+    while (!r && offset < db->map_size) {
- 	const char *p, *q;
- 
- 	/* refresh map, so we see the writes we've just done */
-@@ -2176,7 +2191,7 @@ static int recovery(struct dbengine *db, int flags)
- 	/* bugs in recovery truncates could have left some bogus zeros here */
- 	if (TYPE(ptr) == 0) {
- 	    int orig = offset;
--	    while (TYPE(ptr) == 0 && offset < filesize) {
-+	    while (TYPE(ptr) == 0 && offset < db->map_size) {
- 		offset += 4;
- 		ptr = db->map_base + offset;
- 	    }
-@@ -2189,7 +2204,7 @@ static int recovery(struct dbengine *db, int flags)
- 
- 	/* if this is a commit, we've processed everything in this txn */
- 	if (TYPE(ptr) == COMMIT) {
--	    offset += RECSIZE_safe(db, ptr);
-+	    offset += RECSIZE(ptr);
- 	    continue;
- 	}
- 
-@@ -2203,7 +2218,7 @@ static int recovery(struct dbengine *db, int flags)
- 	}
- 
- 	/* look ahead for a commit */
--	q = db->map_base + filesize;
-+	q = db->map_base + db->map_size;
- 	p = ptr;
- 	for (;;) {
-             if (RECSIZE_safe(db, p) <= 0) {
-@@ -2215,7 +2230,7 @@ static int recovery(struct dbengine *db, int flags)
-                 p = q;
-                 break;
-             }
--	    p += RECSIZE_safe(db, p);
-+	    p += RECSIZE(p);
- 	    if (p >= q) break;
- 	    if (TYPE(p) == COMMIT) break;
- 	}
-@@ -2224,7 +2239,16 @@ static int recovery(struct dbengine *db, int flags)
- 		   "skiplist recovery %s: found partial txn, not replaying",
- 		   db->fname);
- 
--	    filesize = offset;
-+	    /* no commit, we should truncate */
-+	    if (ftruncate(db->fd, offset) < 0) {
-+		syslog(LOG_ERR, 
-+		       "DBERROR: skiplist recovery %s: ftruncate: %m",
-+		       db->fname);
-+		r = CYRUSDB_IOERROR;
-+	    }
-+
-+	    /* set the map size back as well */
-+	    db->map_size = offset;
- 
- 	    break;
- 	}
-@@ -2290,7 +2314,7 @@ static int recovery(struct dbengine *db, int flags)
- 	    }
- 	    offsetnet = htonl(offset);
- 
--	    lvl = LEVEL_safe(db, ptr);
-+	    lvl = LEVEL(ptr);
- 	    if (lvl > SKIPLIST_MAXLEVEL) {
- 		syslog(LOG_ERR,
- 		       "DBERROR: skiplist recovery %s: node claims level %d (greater than max %d)",
-@@ -2299,15 +2323,15 @@ static int recovery(struct dbengine *db, int flags)
- 	    } else {
- 		/* NOTE - in the bogus case where a record with the same key already
- 		 * exists, there are three possible cases:
--		 * lvl == LEVEL_safe(db, keyptr)
-+		 * lvl == LEVEL(keyptr)
- 		 *    * trivial: all to me, all mine to keyptr's FORWARD
--		 * lvl > LEVEL_safe(db, keyptr)	 -
-+		 * lvl > LEVEL(keyptr)	 -
- 		 *    * all updateoffsets values should point to me
--		 *    * up until LEVEL_safe(db, keyptr) set to keyptr's next values
-+		 *    * up until LEVEL(keyptr) set to keyptr's next values
- 		 *      (updateoffsetsi should be keyptr in these cases)
- 		 *      then point all my higher pointers are updateoffsetsi's
- 		 *      FORWARD instead.
--		 * lvl < LEVEL_safe(db, keyptr)
-+		 * lvl < LEVEL(keyptr)
- 		 *    * updateoffsets values up to lvl should point to me
- 		 *    * all mine should point to keyptr's next values
- 		 *    * from lvl up, all updateoffsetsi should point to
-@@ -2319,7 +2343,7 @@ static int recovery(struct dbengine *db, int flags)
- 		 */
- 		for (i = 0; i < lvl; i++) {
- 		    /* set our next pointers */
--		    if (keyptr && i < LEVEL_safe(db, keyptr)) {
-+		    if (keyptr && i < LEVEL(keyptr)) {
-                         /* need to replace the matching record key */
- 			newoffsetsi = 
- 			    htonl(FORWARD(keyptr, i));
-@@ -2338,9 +2362,9 @@ static int recovery(struct dbengine *db, int flags)
- 		lseek(db->fd, FIRSTPTR(ptr) - db->map_base, SEEK_SET);
- 		retry_write(db->fd, (char *) newoffsets, 4 * lvl);
-                 
--		if (keyptr && lvl < LEVEL_safe(db, keyptr)) {
-+		if (keyptr && lvl < LEVEL(keyptr)) {
- 		    uint32_t newoffsetnet;
--		    for (i = lvl; i < LEVEL_safe(db, keyptr); i++) {
-+		    for (i = lvl; i < LEVEL(keyptr); i++) {
- 			newoffsetnet = htonl(FORWARD(keyptr, i));
- 			/* replace 'updateoffsets' to point onwards */
- 			lseek(db->fd, 
-@@ -2356,22 +2380,21 @@ static int recovery(struct dbengine *db, int flags)
- 	}
- 
- 	/* move to next record */
--	unsigned size = RECSIZE_safe(db, ptr);
--	if (!size) break;
--	offset += size;
-+	offset += RECSIZE(ptr);
-     }
- 
--    /* didn't read the exact end?  We should truncate */
--    if (offset < db->map_size) {
--	if (ftruncate(db->fd, offset) < 0) {
--	    syslog(LOG_ERR,
--		   "DBERROR: skiplist recovery %s: ftruncate: %m",
--		   db->fname);
--	    r = CYRUSDB_IOERROR;
--	}
-+    if (libcyrus_config_getswitch(CYRUSOPT_SKIPLIST_ALWAYS_CHECKPOINT)) {
-+	/* refresh map, so we see the writes we've just done */
-+	map_refresh(db->fd, 0, &db->map_base, &db->map_len, db->map_size,
-+		    db->fname, 0);
- 
--	/* set the map size back as well */
--	db->map_size = offset;
-+	r = mycheckpoint(db);
-+
-+	if (r || !(flags & RECOVERY_CALLER_LOCKED)) {
-+	    unlock(db);
-+	}
-+    
-+	return r;
-     }
- 
-     /* fsync the recovered database */
-@@ -2404,16 +2427,13 @@ static int recovery(struct dbengine *db, int flags)
-     }
- 
-     if (!r && need_checkpoint) {
--	/* refresh map, so we see the writes we've just done */
--	map_refresh(db->fd, 0, &db->map_base, &db->map_len, db->map_size,
--		    db->fname, 0);
- 	r = mycheckpoint(db);
-     }
- 
--    if (r || !(flags & RECOVERY_CALLER_LOCKED)) {
-+    if(r || !(flags & RECOVERY_CALLER_LOCKED)) {
- 	unlock(db);
-     }
--
-+    
-     return r;
- }
- 
-diff --git a/lib/imapoptions b/lib/imapoptions
-index fedc695..597e1c4 100644
---- a/lib/imapoptions
-+++ b/lib/imapoptions
-@@ -169,7 +169,7 @@ are listed with ``<none>''.
-    affect LMTP delivery of messages directly to mailboxes via
-    plus-addressing. */
- 
--{ "annotation_db", "twoskip", STRINGLIST("berkeley", "berkeley-hash", "skiplist", "twoskip")}
-+{ "annotation_db", "skiplist", STRINGLIST("berkeley", "berkeley-hash", "skiplist", "twoskip")}
- /* The cyrusdb backend to use for mailbox annotations. */
- 
- { "annotation_db_path", NULL, STRING }
-@@ -508,7 +508,7 @@ Blank lines and lines beginning with ``#'' are ignored.
-    specifies the actual key used for iSchedule DKIM signing within the
-    domain. */
- 
--{ "duplicate_db", "twoskip", STRINGLIST("berkeley", "berkeley-nosync", "berkeley-hash", "berkeley-hash-nosync", "skiplist", "sql", "twoskip")}
-+{ "duplicate_db", "skiplist", STRINGLIST("berkeley", "berkeley-nosync", "berkeley-hash", "berkeley-hash-nosync", "skiplist", "sql", "twoskip")}
- /* The cyrusdb backend to use for the duplicate delivery suppression
-    and sieve. */
- 
-@@ -1013,10 +1013,10 @@ Blank lines and lines beginning with ``#'' are ignored.
- { "maxword", 131072, INT }
- /* Maximum size of a single word for the parser.  Default 128k */
- 
--{ "mboxkey_db", "twoskip", STRINGLIST("berkeley", "skiplist", "twoskip") }
-+{ "mboxkey_db", "skiplist", STRINGLIST("berkeley", "skiplist", "twoskip") }
- /* The cyrusdb backend to use for mailbox keys. */
- 
--{ "mboxlist_db", "twoskip", STRINGLIST("flat", "berkeley", "berkeley-hash", "skiplist", "sql", "twoskip")}
-+{ "mboxlist_db", "skiplist", STRINGLIST("flat", "berkeley", "berkeley-hash", "skiplist", "sql", "twoskip")}
- /* The cyrusdb backend to use for the mailbox list. */
- 
- { "mboxlist_db_path", NULL, STRING }
-@@ -1376,7 +1376,7 @@ If all partitions are over that limit, this feature is not used anymore.
- /* Unix domain socket that ptloader listens on.
-    (defaults to configdir/ptclient/ptsock) */
- 
--{ "ptscache_db", "twoskip", STRINGLIST("berkeley", "berkeley-hash", "skiplist", "twoskip")}
-+{ "ptscache_db", "skiplist", STRINGLIST("berkeley", "berkeley-hash", "skiplist", "twoskip")}
- /* The cyrusdb backend to use for the pts cache. */
- 
- { "ptscache_db_path", NULL, STRING }
-@@ -1401,7 +1401,7 @@ If all partitions are over that limit, this feature is not used anymore.
- /* This specifies the Class Selector or Differentiated Services Code Point
-    designation on IP headers (in the ToS field). */
- 
--{ "quota_db", "twoskip", STRINGLIST("flat", "berkeley", "berkeley-hash", "skiplist", "sql", "quotalegacy", "twoskip")}
-+{ "quota_db", "skiplist", STRINGLIST("flat", "berkeley", "berkeley-hash", "skiplist", "sql", "quotalegacy", "twoskip")}
- /* The cyrusdb backend to use for quotas. */
- 
- { "quota_db_path", NULL, STRING }
-@@ -1521,7 +1521,7 @@ If all partitions are over that limit, this feature is not used anymore.
-    recommended for most cases - it's a good compromise which
-    keeps words separate. */
- 
--{ "seenstate_db", "twoskip", STRINGLIST("flat", "berkeley", "berkeley-hash", "skiplist", "twoskip")}
-+{ "seenstate_db", "skiplist", STRINGLIST("flat", "berkeley", "berkeley-hash", "skiplist", "twoskip")}
- /* The cyrusdb backend to use for the seen state. */
- 
- { "sendmail", "/usr/lib/sendmail", STRING }
-@@ -1715,7 +1715,7 @@ product version in the capabilities */
- { "statuscache", 0, SWITCH }
- /* Enable/disable the imap status cache. */
- 
--{ "statuscache_db", "twoskip", STRINGLIST("berkeley", "berkeley-nosync", "berkeley-hash", "berkeley-hash-nosync", "skiplist", "sql", "twoskip") }
-+{ "statuscache_db", "skiplist", STRINGLIST("berkeley", "berkeley-nosync", "berkeley-hash", "berkeley-hash-nosync", "skiplist", "sql", "twoskip") }
- /* The cyrusdb backend to use for the imap status cache. */
- 
- { "statuscache_db_path", NULL, STRING }
-@@ -1880,7 +1880,7 @@ product version in the capabilities */
- /* File containing the private key belonging to the certificate in
-    tls_server_cert. */
- 
--{ "tls_sessions_db", "twoskip", STRINGLIST("berkeley", "berkeley-nosync", "berkeley-hash", "berkeley-hash-nosync", "skiplist", "sql", "twoskip")}
-+{ "tls_sessions_db", "skiplist", STRINGLIST("berkeley", "berkeley-nosync", "berkeley-hash", "berkeley-hash-nosync", "skiplist", "sql", "twoskip")}
- /* The cyrusdb backend to use for the TLS cache. */
- 
- { "tls_sessions_db_path", NULL, STRING }
-@@ -1944,7 +1944,7 @@ product version in the capabilities */
-    this user.  NOTE: This must be an existing local user name with an
-    INBOX, NOT an email address! */
- 
--{ "zoneinfo_db", "twoskip", STRINGLIST("flat", "berkeley", "berkeley-hash", "skiplist", "twoskip")}
-+{ "zoneinfo_db", "skiplist", STRINGLIST("flat", "berkeley", "berkeley-hash", "skiplist", "twoskip")}
- /* The cyrusdb backend to use for zoneinfo. */
- 
- { "zoneinfo_db_path", NULL, STRING }

cyrus-imapd-2.5.tar.gz Changed

cyrus-imapd.cvt_cyrusdb_all Changed

@@ -1,252 +1,418 @@
 #!/bin/bash
-# Simplified version of Simon Matter's cvt_cyrusdb_all.
 
-function usage() {
-    echo "Usage: $0 -C <altconfig> --verbose --devel <export|import>"
-    exit 1
-}
-
- -f "/etc/sysconfig/cyrus-imapd"  && \
-    . /etc/sysconfig/cyrus-imapd
-
- -f "/etc/default/cyrus-imapd"  && \
-    . /etc/default/cyrus-imapd
-
-# Ensure this script is run as the cyrus user.
-if  -x "$(which runuser 2>/dev/null)" ; then
-    SWITCHUSER=runuser
-else
-    SWITCHUSER=su
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program; if not, write to the Free Software
+# Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA  02111-1307  USA
+
+# This script converts all db files of a cyrus installation from their
+# existing format to the format required by the current installation.
+# The format of current db files is determined using the 'file' command
+# with a magic file added for skiplist db, the new format is read from
+# a config file usually in /usr/share/cyrus-imapd/rpm/db.cfg, which is
+# created while compiling. After converting, the db.cfg file is
+# copied to a cache file usually at /var/lib/imap/rpm/db.cfg.cache to
+# allow bypassing this converting script if both files are identical.
+# While this is a bit less secure, it may be useful on big server where
+# db converting is done automatically.
+#
+# This script can safely be run as root, it will reexec itself as user
+# cyrus if needed.
+#
+# author: Simon Matter, Invoca Systems <simon.matter@invoca.ch>
+
+# changelog
+# v1.0.1, Oct 22 2002 Simon Matter <simon.matter@invoca.ch>
+# - added two-step conversion method
+#
+# v1.0.2, Jan 10 2003 Simon Matter <simon.matter@invoca.ch>
+# - fixed a bug where cvt_cyrusdb was called to convert empty or
+#   nonexistent files
+#
+# v1.0.3, Mar 14 2003 Simon Matter <simon.matter@invoca.ch>
+# - fixed a problem with new versions of the file command
+#
+# v1.0.4
+# - added GPL license
+#
+# v1.0.5, May 02 2003 Simon Matter <simon.matter@invoca.ch>
+# - modified exec path
+#
+# v1.0.6, Jul 18 2003 Simon Matter <simon.matter@invoca.ch>
+# - changed db3 to berkeley
+# - added new db backends for 2.2
+#
+# v1.0.7, Jan 23 2004 Simon Matter <simon.matter@invoca.ch>
+# - included some modifications from Luca Olivetti <luca@olivetti.cjb.net>
+# - added masssievec functionality
+#
+# v1.0.8, Jan 28 2004 Simon Matter <simon.matter@invoca.ch>
+# - convert sieve scripts to UTF-8 before calling masssievec
+#
+# v1.0.9, Jan 29 2004 Simon Matter <simon.matter@invoca.ch>
+# - convert sieve scripts to UTF-8 only if sievec failed before
+#
+# v1.0.10, Feb 24 2004 Simon Matter <simon.matter@invoca.ch>
+# - change su within init script to get input from
+#   /dev/null, this prevents hang when running in SELinux
+#
+# v1.0.11, Mar 02 2004 Simon Matter <simon.matter@invoca.ch>
+# - fixed SELinux fix
+#
+# v1.0.12, Dec 16 2004 Simon Matter <simon.matter@invoca.ch>
+# - use runuser instead of su if available
+#
+# v1.0.13, Jul 15 2005 Simon Matter <simon.matter@invoca.ch>
+# - don't use flat in the two step conversion, use skiplist instead
+#
+# v1.0.14, Jul 18 2005 Simon Matter <simon.matter@invoca.ch>
+# - replace the order of the magic files in the file call to make
+#   sure skiplist is detected correctly.
+#
+# v1.0.15, Aug 17 2005 Simon Matter <simon.matter@invoca.ch>
+# - add functionality to export all berkeley db files to skiplist
+#
+# v1.1.0, Aug 18 2005 Simon Matter <simon.matter@invoca.ch>
+# - fix export functionality, try to recover Berkeley databases
+#   as much as possible before any conversion.
+#
+# v1.1.1, Dec 05 2005 Simon Matter <simon.matter@invoca.ch>
+# - run db_checkpoint in background with a timeout to prevent
+#   that cyrus-imapd doesn't start at all if it hangs.
+#
+# v1.1.2, Dec 06 2005 Simon Matter <simon.matter@invoca.ch>
+# - make handling of db_checkpoint more robust
+#
+# v1.2.0, Jan 12 2006 Simon Matter <simon.matter@invoca.ch>
+# - adopt for cyrus-imapd-2.3
+#
+# v1.2.1, Jan 13 2006 Simon Matter <simon.matter@invoca.ch>
+# - code cleanup
+#
+# v1.2.2, Nov 29 2007 Simon Matter <simon.matter@invoca.ch>
+# - add ability to handle "@include" options in imapd.conf, patch
+#   provided by Tim Bannister
+#
+# v1.2.3, Feb 07 2008 Simon Matter <simon.matter@invoca.ch>
+# - add ability to handle tabs in imapd.conf, patch provided
+#   by Franz Knipp
+# - disable default values for some config options like sievedir
+#
+# v1.2.4, Apr 23 2008 Simon Matter <simon.matter@invoca.ch>
+# - add support for statuscache.db
+#
+# v1.3.0, Sep 29 2008 Simon Matter <simon.matter@invoca.ch>
+# - add multi-instance support
+#
+# v1.3.1, Oct 09 2008 Simon Matter <simon.matter@invoca.ch>
+# - improve variable handling
+#
+# v1.3.2, May 26 2009 Simon Matter <simon.matter@invoca.ch>
+# - add some sanity checks to multi-instance support
+#
+# v1.3.3, May 27 2009 Simon Matter <simon.matter@invoca.ch>
+# - make some cosmetic changes
+#
+# v1.3.4, Dec 22 2009 Simon Matter <simon.matter@invoca.ch>
+# - add support for user_deny.db
+#
+# v1.3.5, Mar  6 2014 Jeroen van Meeuwen <vanmeeuwen@kolabsys.com>
+# - use default database configuration values if not specified in
+#   configuration
+
+VERSION=1.3.5
+
+ -f /etc/sysconfig/cyrus-imapd${INSTANCE}  && \
+    . /etc/sysconfig/cyrus-imapd${INSTANCE}
+
+ "${QUICK}" == "1"  && exit 0
+
+PIDFILE=/var/run/cyrus-master${INSTANCE}.pid
+
+# instance config
+CYRUSCONF=/etc/cyrus${INSTANCE}.conf
+IMAPDCONF=/etc/imapd${INSTANCE}.conf
+
+# make sure what we have is a valid instance
+# and that config files are present
+if  -n "$INSTANCE" ; then
+   -L /etc/rc.d/init.d/${BASENAME}  || exit 0
 fi
-
-#if  ! $(whoami) = "cyrus" ; then
-#    exec $SWITCHUSER -s /bin/bash - cyrus -c "cd \$HOME < /dev/null ; INSTANCE=$0 $*"
-#    exit 0
-#fi
-
-umask 166
-
-alt_config="/etc/imapd.conf"
-devel_mode=0
-verbose_mode=0
-mode=""
-
-while  $# -gt 0 ; do
-    case $1 in
-        -C)
-            alt_config=$2
-            shift; shift
-        ;;
-
-        --devel)
-            devel_mode=1
-            shift
-        ;;
-
-        --verbose)
-            verbose_mode=1
-            shift
-        ;;
-
-        export|import)
-            mode=$1
-            shift
-        ;;
-
-        *)
-            usage
-        ;;
-
-    esac
-done
-
-if  -z "${mode}" ; then
-    usage
+ -f $CYRUSCONF  || exit 0
+ -f $IMAPDCONF  || exit 0
+
+if  -f $PIDFILE ; then
+  read CYRUS_PID < $PIDFILE
+  if  -n "$CYRUS_PID" ; then
+    if ps -p $CYRUS_PID > /dev/null 2>&1; then
+      echo "ERROR: cyrus-master is running, unable to convert mailboxes!"
+      exit 1
+    fi
+  fi
 fi
 
-if  ${devel_mode} -eq 1 ; then
-    verbose_mode=1
+if  ! -f $IMAPDCONF ; then
+  echo "ERROR: configuration file '${IMAPDCONF}' not found, exiting!"
+  exit 1
 fi
 
-declare -a dbs
-declare -a db_paths
-
-dbs${#dbs@}="annotation_db";    db_paths${#db_paths@}="annotations.db"
-dbs${#dbs@}="duplicate_db";     db_paths${#db_paths@}="deliver.db"
-# dbs${#dbs@}="mboxkey_db";       db_paths${#db_paths@}=""
-dbs${#dbs@}="mboxlist_db";      db_paths${#db_paths@}="mailboxes.db"
-dbs${#dbs@}="ptscache_db";      db_paths${#db_paths@}="ptclient/ptscache.db"
-dbs${#dbs@}="quota_db";         db_paths${#db_paths@}="quotas.db"
-# dbs${#dbs@}="seenstate_db";     db_paths${#db_paths@}=""
-dbs${#dbs@}="statuscache_db";   db_paths${#db_paths@}="statuscache.db"
-# dbs${#dbs@}="subscription_db";  db_paths${#db_paths@}=""
-dbs${#dbs@}="tls_sessions_db";  db_paths${#db_paths@}="tls_sessions.db"
-dbs${#dbs@}="userdeny_db";      db_paths${#db_paths@}="user_deny.db"
-
-# /usr/lib/cyrus-imapd/cyr_dbtool /var/lib/imap/annotations.db skiplist consistent; echo $?
-# Yes, consistent
-# 0
-
-cvt_cyrusdb="/usr/lib/cyrus-imapd/cvt_cyrusdb"
-cyr_dbtool="/usr/lib/cyrus-imapd/cyr_dbtool"
-cyr_info="/usr/lib/cyrus-imapd/cyr_info"
-
-system_magic=$(file --version 2>&1 | awk '/magic file/ {print $4}')
-
-if  ${devel_mode} -gt 0 ; then
-    if  -f "$(pwd)/imap/cvt_cyrusdb" ; then
-        cvt_cyrusdb="$(pwd)/imap/cvt_cyrusdb"
-    fi
-
-    if  -f "$(pwd)/imap/.libs/cyr_dbtool" ; then
-        cyr_dbtool="$(pwd)/imap/cyr_dbtool"
-    fi
-
-    if  -f "$(pwd)/imap/.libs/cyr_info" ; then
-        cyr_info="$(pwd)/imap/cyr_info"
-    fi
+# fallback to su if runuser not available
+if  -x /sbin/runuser ; then
+  RUNUSER=runuser
+else
+  RUNUSER=su
 fi
 
-function check_consistency() {
-    db_path=$1
-    db_tech=$2
-
-    if  ! -f "${db_path}" ; then
-        return 1
-    fi
+# force cyrus user for security reasons
+if  ! $(whoami) = "cyrus" ; then
+  exec $RUNUSER - cyrus -c "cd $PWD < /dev/null ; INSTANCE=$INSTANCE $0 $*"
+fi
 
-    retval=$(${cyr_dbtool} -C ${alt_config} ${db_path} ${db_tech} consistent >/dev/null 2>&1; echo $?)
+# special function for migration
+EXPORT=$1
 
-    return ${retval}
-}
+# files get mode 0600
+umask 166
 
-function convert_database() {
-    retval=$(${cvt_cyrusdb} -C ${alt_config} "$1" "$2" "$3" "$4" >/dev/null 2>&1; echo $?)
-    chown cyrus:mail "$3"
-}
+# show version info in log files
+echo "cvt_cyrusdb_all version: $VERSION"
 
-function expand_db_path() {
-    if  "${1:0:1}" != "/" ; then
-        config_directory=$(get_config "configdirectory")
-        value="${config_directory}/${1}"
+# expand_config <path>
+# handle "@include" sections from imapd style config file
+expand_config() {
+  while read line; do
+    if printf "%s\n" "${line}" | grep -q '^@include:'; then
+      expand_config "$( printf "%s\n" "${line}" | cut -d : -f 2- | sed -e 's/^\t *//' )"
+    else
+      printf "%s\n" "${line}"
     fi
-
-    echo "${value}"
+  done < $1
 }
 
-function get_config() {
-    echo $(${cyr_info} -C ${alt_config} conf | grep -E "^$1:\s+" | sed -r -e 's/a-z0-9_-+\:\s*(.*)/\1/g')
+# get_config <config> <default>
+# extracts config option from config file
+get_config() {
+  searchstr=$1
+  if config="$(expand_config $IMAPDCONF | egrep "^${searchstr}:")"; then
+    CFGVAL="$(printf "%s\n" "$config" | cut -d : -f 2- | sed -e 's/^\t *//')"
+  else
+    if  -z "$2" ; then
+      echo "ERROR: config option '$1' not found in ${IMAPDCONF}, exiting!" 1>&2
+      return 1
+    fi
+    CFGVAL="$2"
+  fi
+  echo "get_config ${1}: $CFGVAL" 1>&2
+  echo "$CFGVAL"
 }
 
-function get_config_default() {
-    echo $(${cyr_info} -C ${alt_config} conf-default | grep -E "^$1:\s+" | sed -r -e 's/a-z0-9_-+\:\s*(.*)/\1/g')
+# where to find files and directories
+data_dir=/usr/share/cyrus-imapd/rpm
+lib_dir=/usr/lib/cyrus-imapd
+system_magic=$(file --version | awk '/magic file/ {print $4}')
+cyrus_magic=${data_dir}/magic
+cvt_cyrusdb=${lib_dir}/cvt_cyrusdb
+sievec=${lib_dir}/sievec
+masssievec=${lib_dir}/masssievec
+imap_prefix=$(get_config configdirectory) || exit 1
+sieve_dir=$(get_config sievedir) || exit 1
+db_cfg=${data_dir}/db.cfg
+db_current=${imap_prefix}/rpm/db.cfg.current
+db_cache=${imap_prefix}/rpm/db.cfg.cache
+
+# source default db backend config
+. $db_cfg
+
+# get configured db backend config
+duplicate_db=$(get_config duplicate_db $duplicate_db) || duplicate_db="skiplist"
+mboxlist_db=$(get_config mboxlist_db $mboxlist_db) || mboxlist_db="skiplist"
+seenstate_db=$(get_config seenstate_db $seenstate_db) || seenstate_db="skiplist"
+subscription_db=$(get_config subscription_db $subscription_db) || subscription_db="skiplist"
+tlscache_db=$(get_config tlscache_db $tlscache_db) || tlscache_db="skiplist"
+annotation_db=$(get_config annotation_db $annotation_db) || annotation_db="skiplist"
+mboxkey_db=$(get_config mboxkey_db $mboxkey_db) || mboxkey_db="skiplist"
+ptscache_db=$(get_config ptscache_db $ptscache_db) || ptscache_db="skiplist"
+quota_db=$(get_config quota_db $quota_db) || quota_db="quota_legacy"
+statuscache_db=$(get_config statuscache_db $statuscache_db) || statuscache_db="skiplist"
+userdeny_db=$(get_config userdeny_db $userdeny_db) || userdeny_db="skiplist"
+
+# remember current db backend config
+{
+echo "duplicate_db=${duplicate_db}"
+echo "mboxlist_db=${mboxlist_db}"
+echo "seenstate_db=${seenstate_db}"
+echo "subscription_db=${subscription_db}"
+echo "tlscache_db=${tlscache_db}"
+echo "annotation_db=${annotation_db}"
+echo "mboxkey_db=${mboxkey_db}"
+echo "ptscache_db=${ptscache_db}"
+echo "quota_db=${quota_db}"
+echo "statuscache_db=${statuscache_db}"
+echo "userdeny_db=${userdeny_db}"
+echo "sieve_version=${sieve_version}"
+} | sort > $db_current
+
+# file_type <file>
+file_type() {
+  this_type=$(file -b -m "${cyrus_magic}:${system_magic}" "$1" 2> /dev/null)
+  if echo "$this_type" | grep -qi skip > /dev/null 2>&1; then
+    echo skiplist
+  elif echo "$this_type" | grep -qi text > /dev/null 2>&1; then
+    echo flat
+  else
+    echo berkeley
+  fi
 }
 
-function get_current() {
-    default_path="$2"
-    value=$(${cyr_info} -C ${alt_config} conf-all | grep -E "^$1_path:\s+" | sed -r -e 's/a-z0-9_-+\:\s*(.*)/\1/g')
-
-    if  -z "${value}" ; then
-        if  ! -z "${default_path}" ; then
-            value=$(expand_db_path "${default_path}")
+# cvt_file <file> <db>
+cvt_file() {
+  target="$1"
+  new_db="$2"
+  if  -s "$target" ; then
+    old_db=$(file_type "$target")
+    if  ! "$old_db" = "$new_db" ; then
+      # The two-step conversion is paranoia against the filenames being encoded
+      # inside the database or logfiles (berkeley does this, for example).
+      rm -f "${target}.skiplist"
+      if  "$old_db" = "skiplist" ; then
+        cp -a "$target" "${target}.skiplist"
+      else
+        $cvt_cyrusdb -C $IMAPDCONF "$target" "$old_db" "${target}.skiplist" skiplist
+      fi
+      RETVAL=$?
+      ERRVAL=$(( $ERRVAL + $RETVAL ))
+      if  $RETVAL -eq 0 ; then
+        rm -f "$target"
+        if  -s "${target}.skiplist" ; then
+          if  "$new_db" = "skiplist" ; then
+            cp -a "${target}.skiplist" "$target"
+          else
+            $cvt_cyrusdb -C $IMAPDCONF "${target}.skiplist" skiplist "$target" "$new_db"
+          fi
         fi
+        RETVAL=$?
+        ERRVAL=$(( $ERRVAL + $RETVAL ))
+        if  $RETVAL -eq 0 ; then
+          rm -f "${target}.skiplist"
+        else
+          echo "ERROR: unable to convert ${target}.skiplist from skiplist to $new_db"
+        fi
+      else
+        echo "ERROR: unable to convert $target from $old_db to skiplist"
+      fi
     fi
+  fi
+}
 
-    if  -z "${value}" ; then
-        return 1
-    fi
-
-    if  ! -f "${value}" ; then
-        return 1
-    fi
-
-    current=$(file -b -m /var/lib/imap/rpm/magic:${system_magic} "${value}")
-
-    if echo "${current}" | grep -qi skiplist > /dev/null 2>&1; then
-        current="skiplist"
-    elif echo "${current}" | grep -qi twoskip > /dev/null 2>&1; then
-        current="twoskip"
-    elif echo "${current}" | grep -qi text > /dev/null 2>&1; then
-        current="flat"
-    else
-        current="berkeley"
+# cvt_to_utf8 <file>
+cvt_to_utf8() {
+  target="$1"
+  if  -s "$target" ; then
+    if ! $sievec -C $IMAPDCONF "$target" "${target}.sievec"; then
+      iconv --from-code=ISO-8859-1 --to-code=UTF-8 --output="${target}.UTF-8" "$target"
+      if  -s "${target}.UTF-8" ; then
+        # preserve timestamp
+        touch --reference="${target}" "${target}.UTF-8"
+        mv -f "${target}.UTF-8" "$target"
+      else
+        ERRVAL=$(( $ERRVAL + 1 ))
+      fi
     fi
-
-    echo "${current}"
+    rm -f "${target}.sievec"
+  fi
 }
 
-config_directory=$(get_config "configdirectory")
-
-if  ${devel_mode} -eq 1 ; then
-    echo "Using configuration directory: '${config_directory}'"
+ERRVAL=0
+
+# make sure our Berkeley databases are in a sane state
+# wait for db_checkpoint to end successfully or kill it after a timeout
+db_checkpoint -v -1 -h ${imap_prefix}/db &
+DB_CHECK_PID=$!
+CNT=0
+while  $CNT -lt 60 ; do
+  if ! kill -0 $DB_CHECK_PID > /dev/null 2>&1; then
+    break
+  fi
+  sleep 1
+  let CNT+=1
+done
+if kill -0 $DB_CHECK_PID > /dev/null 2>&1; then
+  kill -USR1 $DB_CHECK_PID > /dev/null 2>&1
+  sleep 1
+  kill -KILL $DB_CHECK_PID > /dev/null 2>&1
+  wait $DB_CHECK_PID > /dev/null 2>&1
 fi
 
-x=0
-while  ${x} -lt ${#dbs@} ; do
-    config=$(get_config "${dbs${x}}")
-    default=$(get_config_default "${dbs${x}}")
-    current=$(get_current "${dbs${x}}" "${db_paths${x}}")
-
-    if  ${verbose_mode} -eq 1 ; then
-        echo -n "${dbs${x}}: "
-    fi
-
-    if  $? -ne 0 ; then
-        if  ${verbose_mode} -eq 1 ; then
-            echo "SKIPPED"
-        fi
-        let x++
-        continue
-    fi
-
-    # If configuration doesn't hold anything, we have defaults
-    if  -z "${config}" ; then
-        config="${default}"
-    fi
-
-    if  "${config}" == "${default}" -a "${config}" == "${current}" ; then
-        if  ${verbose_mode} -eq 1 ; then
-            echo "SKIPPED (unchanged)"
-        fi
-        let x++
-        continue
-    fi
-
-    db_path=$(expand_db_path "${db_paths${x}}")
-
-    if  ! -f "${db_path}" ; then
-        if  ${verbose_mode} -eq 1 ; then
-            echo "SKIPPED (No such file or directory '${db_path}')"
-        fi
-        let x++
-        continue
-    fi
-
-    if  "${current}" != "${default}" ; then
-        if  ${devel_mode} -eq 1 ; then
-            echo -n "to default(1), "
-        fi
-        convert_database "${db_path}" "${current}" "${db_path}.${default}" "${default}"
-    fi
-
-    if  "${current}" != "${config}" ; then
-        mv "${db_path}" "${db_path}.${current}"
-        if  "${current}" != "${default}" ; then
-            if  ${devel_mode} -eq 1 ; then
-                echo -n "to default(2), "
-            fi
-            convert_database "${db_path}.${current}" "${current}" "${db_path}.${default}" "${default}"
-        fi
+# do a normal recovery
+db_recover -v -h ${imap_prefix}/db
+RETVAL=$?
+if  $RETVAL -ne 0 ; then
+  # try a catastrophic recovery instead of normal recovery
+  db_recover -v -c -h ${imap_prefix}/db
+  RETVAL=$?
+  ERRVAL=$(( $ERRVAL + $RETVAL ))
+  if  $RETVAL -ne 0 ; then
+    echo "ERROR: catastrophic recovery of Berkeley databases failed"
+  fi
+fi
 
-        if  ${devel_mode} -eq 1 ; then
-            echo -n "to config(1), "
-        fi
-        convert_database "${db_path}.${current}" "${current}" "${db_path}" "${config}"
-    fi
+if  "$EXPORT" = "export" ; then
+  # convert all db files to portable format for migration
+  # TODO: quota_db, we don't touch it for now
+  cvt_file ${imap_prefix}/deliver.db           "skiplist"
+  cvt_file ${imap_prefix}/mailboxes.db         "skiplist"
+  cvt_file ${imap_prefix}/tls_sessions.db      "skiplist"
+  cvt_file ${imap_prefix}/annotations.db       "skiplist"
+  cvt_file ${imap_prefix}/ptclient/ptscache.db "skiplist"
+  cvt_file ${imap_prefix}/statuscache.db       "skiplist"
+  cvt_file ${imap_prefix}/user_deny.db         "flat"
+  rm -vf ${imap_prefix}/db/log.*
+  rm -vf ${imap_prefix}/db/__db.*
+else
+  # always convert db files which have been converted to skiplist
+  # TODO: quota_db, we don't touch it for now
+  cvt_file ${imap_prefix}/deliver.db           "$duplicate_db"
+  cvt_file ${imap_prefix}/mailboxes.db         "$mboxlist_db"
+  cvt_file ${imap_prefix}/tls_sessions.db      "$tlscache_db"
+  cvt_file ${imap_prefix}/annotations.db       "$annotation_db"
+  cvt_file ${imap_prefix}/ptclient/ptscache.db "$ptscache_db"
+  cvt_file ${imap_prefix}/statuscache.db       "$statuscache_db"
+  cvt_file ${imap_prefix}/user_deny.db         "$userdeny_db"
+  # do we have to convert all databases?
+  if ! cmp -s $db_current $db_cache; then
+    # we treat sieve scripts the same way like db files
+    find ${sieve_dir}/ -name "*.script" -type f | while read db_file trash; do
+      cvt_to_utf8 "$db_file"
+    done
+    $masssievec $sievec $IMAPDCONF
+    # convert all db files left
+    find ${imap_prefix}/user/ -name "*.seen" -type f | while read db_file trash; do
+      cvt_file "$db_file" "$seenstate_db"
+    done
+    find ${imap_prefix}/user/ -name "*.sub" -type f | while read db_file trash; do
+      cvt_file "$db_file" "$subscription_db"
+    done
+    find ${imap_prefix}/user/ -name "*.mboxkey" -type f | while read db_file trash; do
+      cvt_file "$db_file" "$mboxkey_db"
+    done
+  fi
+fi
 
-    if  ${verbose_mode} -eq 1 ; then
-        echo "OK"
-    fi
+# update the config cache file so we can check whether something has changed
+if  $ERRVAL -eq 0 ; then
+  mv -f $db_current $db_cache
+else
+  rm -f $db_cache
+  rm -f $db_current
+fi
 
-    let x++
-done
+exit $ERRVAL

cyrus-imapd.dsc Changed

cyrus-imapd.init Changed

cyrus-imapd.magic Changed

debian.changelog Changed

@@ -1,110 +1,3 @@
-cyrus-imapd (2.5~dev2014120901-0~kolab1) unstable; urgency=low
-
-  * New snapshot at f01310085cbf58958412deda9c61ecfd7c82a19b
-
- -- Jeroen van Meeuwen (Kolab Systems) <vanmeeuwen@kolabsys.com>  Tue,  9 Dec 2014 11:30:13 +0100
-
-cyrus-imapd (2.5~dev2014120401-0~kolab1) unstable; urgency=low
-
-  * New snapshot at 7f5b99bab9641d89735a882761e4d16eca3a4068
-
- -- Jeroen van Meeuwen (Kolab Systems) <vanmeeuwen@kolabsys.com>  Thu,  4 Dec 2014 11:30:13 +0100
-
-cyrus-imapd (2.5~dev2014112701-0~kolab1) unstable; urgency=low
-
-  * New snapshot at 3123ee085528330cd7045a0ba4f9296be77cb5ac
-
- -- Jeroen van Meeuwen (Kolab Systems) <vanmeeuwen@kolabsys.com>  Thu, 27 Nov 2014 11:30:13 +0100
-
-cyrus-imapd (2.5~dev2014112301-0~kolab1) unstable; urgency=low
-
-  * New snapshot at 631aa71cf5fe686563ae1d7e740a5cc76d0bf75f
-
- -- Jeroen van Meeuwen (Kolab Systems) <vanmeeuwen@kolabsys.com>  Sun, 23 Nov 2014 11:30:13 +0100
-
-cyrus-imapd (2.5~dev2014112201-0~kolab1) unstable; urgency=low
-
-  * New snapshot at 790df515eedd8be0050b14bfb7ab471c6ab9c6d3
-
- -- Jeroen van Meeuwen (Kolab Systems) <vanmeeuwen@kolabsys.com>  Sat, 22 Nov 2014 11:30:13 +0100
-
-cyrus-imapd (2.5~dev2014111801-0~kolab1) unstable; urgency=low
-
-  * New snapshot at e1f53f2dda607c19c7f6b2293aba65e4d2c9c47f
-
- -- Jeroen van Meeuwen (Kolab Systems) <vanmeeuwen@kolabsys.com>  Tue, 18 Nov 2014 11:30:13 +0100
-
-cyrus-imapd (2.5~dev2014111701-0~kolab1) unstable; urgency=low
-
-  * New snapshot at b20b142c6ce0ee58e4e47da84a2ce099310b2dcb
-
- -- Jeroen van Meeuwen (Kolab Systems) <vanmeeuwen@kolabsys.com>  Mon, 17 Nov 2014 11:30:13 +0100
-
-cyrus-imapd (2.5~dev2014111201-0~kolab2) unstable; urgency=low
-
-  * Set the default database format to skiplist
-  * New snapshot at 7361099c548f11f5dff5d65583ba39f150e46938
-
- -- Jeroen van Meeuwen (Kolab Systems) <vanmeeuwen@kolabsys.com>  Thu, 12 Nov 2014 11:30:13 +0100
-
-cyrus-imapd (2.5~dev2014111201-0~kolab1) unstable; urgency=low
-
-  * Ship a GIT development snapshot from git.cyrusimap.org
-    58e1f4b277586e448f726bdafe6f2e250c9ea5f4
-
- -- Jeroen van Meeuwen (Kolab Systems) <vanmeeuwen@kolabsys.com>  Wed, 11 Nov 2014 11:30:13 +0100
-
-cyrus-imapd (2.5~dev2014110501-0~kolab1) unstable; urgency=low
-
-  * Ship a GIT development snapshot from git.cyrusimap.org
-    2941de1350adf7f0d99443ace884f40c754df193
-
- -- Jeroen van Meeuwen (Kolab Systems) <vanmeeuwen@kolabsys.com>  Wed,  5 Nov 2014 11:30:13 +0100
-
-cyrus-imapd (2.5~dev2014103002-0~kolab1) unstable; urgency=low
-
-  * Ship a GIT development snapshot from git.cyrusimap.org
-    8f15dcd0c71aff01f6d30e83644e9efcb5855b87
-
- -- Jeroen van Meeuwen (Kolab Systems) <vanmeeuwen@kolabsys.com>  Thu, 30 Oct 2014 11:30:13 +0100
-
-cyrus-imapd (2.5~dev2014101701-0~kolab1) unstable; urgency=low
-
-  * Ship a GIT development snapshot from git.cyrusimap.org
-    9704e5d41981a18f46a63af78cf02c63fe3824a6
-
- -- Jeroen van Meeuwen (Kolab Systems) <vanmeeuwen@kolabsys.com>  Fri, 17 Oct 2014 11:30:13 +0100
-
-cyrus-imapd (2.5~dev2014100701-0~kolab1) unstable; urgency=low
-
-  * Ship a GIT development snapshot from git.cyrusimap.org
-    d8beae4e3a9490b0435caee22ddae313b8f4b17b
-
- -- Jeroen van Meeuwen (Kolab Systems) <vanmeeuwen@kolabsys.com>  Tue,  7 Oct 2014 11:30:13 +0100
-
-cyrus-imapd (2.5~dev2014100301-0~kolab1) unstable; urgency=low
-
-  * Ship a GIT development snapshot from git.cyrusimap.org
-    44e8e7f7d0e16cc6005441709fc7c02e7032d6b3
-
- -- Jeroen van Meeuwen (Kolab Systems) <vanmeeuwen@kolabsys.com>  Fri,  3 Oct 2014 11:30:13 +0100
-
-cyrus-imapd (2.5~dev2014092501-0~kolab1) unstable; urgency=low
-
-  * Ship a GIT development snapshot from git.cyrusimap.org
-    1907cfc24b5b3bee0e4f81f87ba5ab3be3a8a418
-  * Fix event notifications
-
- -- Jeroen van Meeuwen (Kolab Systems) <vanmeeuwen@kolabsys.com>  Thu, 25 Sep 2014 11:30:13 +0100
-
-cyrus-imapd (2.5~dev2014082901-0~kolab1) unstable; urgency=low
-
-  * Ship a GIT development snapshot from git.cyrusimap.org
-    b2ef80bee2102d0eadf280c6c23b5759920d1752
-  * Merge enhanced Sieve Date and Index extension
-
- -- Jeroen van Meeuwen (Kolab Systems) <vanmeeuwen@kolabsys.com>  Fri, 29 Aug 2014 11:30:13 +0100
-
 cyrus-imapd (2.5~dev2014081301-0~kolab1) unstable; urgency=low
 
   * Ship a GIT development snapshot from git.cyrusimap.org

debian.control Changed

debian.series Changed

Build Results
RPM Lint

Refresh

Comments for request 813 1
Comments for request 782 2

dhoffend wrote almost 10 years ago

Build's are okay, tested compiled result on debian 7, fix is working

timotheus wrote almost 10 years ago

This is a huge change. I am anxious to put my vote on it. Do we want such a huge change on the Kolab Updates channel? Or better wait for Kolab 3.4, and the new version will fix it? If you want a certain bug to be fixed, I guess it is better to find a single patch that fixes it, and apply that only, instead of upgrading the full package?

dhoffend wrote almost 10 years ago

The package works for me. But as timo stated. I don't know what else has changed in this cyrus release. ATM I can only guess. Sure a dedicated patch would be the best solution.

On the other hand, like timo said: 3.3 is around 5 month old and 3.4 is coming next month. If this segfault would be a big dealbreaker for many other users the voices would be much louder.

My guess is: I would rather spend time testing Kolab:Dev to prepare for a clean 3.4 release instead of backporting software that's supposed for the upcoming release.

Good question

Request History

dhoffend created request almost 10 years ago

Applied patch to fix ctl_mboxlist segfault

dhoffend accepted request almost 10 years ago

Please add a comment