\documentclass[11pt]{article}
\usepackage{latexsym}
\usepackage{amsmath}
\usepackage{amssymb}
\usepackage{amsthm}
\usepackage{epsfig}
\usepackage{psfig}

\newcommand{\handout}[5]{
  \noindent
  \begin{center}
  \framebox{
    \vbox{
      \hbox to 5.78in { {\bf 6.851: Advanced Data Structures } \hfill #2 }
      \vspace{4mm}
      \hbox to 5.78in { {\Large \hfill #5  \hfill} }
      \vspace{2mm}
      \hbox to 5.78in { {\em #3 \hfill #4} }
    }
  }
  \end{center}
  \vspace*{4mm}
}

\newcommand{\lecture}[4]{\handout{#1}{#2}{#3}{Scribe: #4}{Lecture #1}}

\newtheorem{theorem}{Theorem}
\newtheorem{corollary}[theorem]{Corollary}
\newtheorem{lemma}[theorem]{Lemma}
\newtheorem{observation}[theorem]{Observation}
\newtheorem{proposition}[theorem]{Proposition}
\newtheorem{definition}[theorem]{Definition}
\newtheorem{claim}[theorem]{Claim}
\newtheorem{fact}[theorem]{Fact}
\newtheorem{assumption}[theorem]{Assumption}

% 1-inch margins, from fullpage.sty by H.Partl, Version 2, Dec. 15, 1988.
\topmargin 0pt
\advance \topmargin by -\headheight
\advance \topmargin by -\headsep
\textheight 8.9in
\oddsidemargin 0pt
\evensidemargin \oddsidemargin
\marginparwidth 0.5in
\textwidth 6.5in

\parindent 0in
\parskip 1.5ex
%\renewcommand{\baselinestretch}{1.25}

\begin{document}

\lecture{6 --- February 28, 2007}{Spring 2007}{Prof.\ Erik Demaine}{Matthew Hofmann}

\section{Overview}

In the last lecture we introduced Euler tour trees \cite{euler}, dynamic data structures that can perform link-cut tree operations 
in $O(\lg n)$ time.  We then showed how to implement an efficient dynamic connectivity algorithm using a spanning forest of 
Euler tour trees, as demonstrated in \cite{dyn}.  This yielded an amortized time bound of $O(\lg^{2} n)$ for update operations (such 
as edge insertion and deletion), and $O(\lg n / \lg \lg n)$ for querying the connectivity of two vertices.  In this lecture, we switch 
to examining the lower bound of dynamic connectivity algorithms.  Until recently, the best lower bound for dynamic connectivity
operations was $\Omega(\lg n / \lg \lg n)$, as described by Fredman and Henzinger in \cite{old1} and independently by Miltersen in \cite{old2}.  However, we will show below that it is possible to prove $\Omega(\lg n)$, using the method given by P\v{a}tra\c{s}cu and Demaine in \cite{dc}.

\section{Cell Probe Complexity Model}

The $\Omega(\lg n)$ bound relies on a model of computation called the \emph{cell probe complexity model}, originally described in the context of proving dynamic lower bounds by Fredman and Saks in \cite{ccp}.  The cell probe model views a data structure as a sequence of 
\emph{cells}, or words, each containing a $w$-bit field.  The model calculates the complexity of an algorithm by counting the number of 
reads and writes to the cells; any additional computation is free.  This makes the model comparable to a RAM model with constant time 
access.  Because computation is free, the model is not useful for determining upper bounds, only lower bounds.  

We empirically assume that the size of each cell, $w$, is at least $\lg n$.  This is because we would like the cells to store pointers 
to each of our $n$ vertices, and information theory tells us that we need $\lg n$ bits to address $n$ items.  For the the following proof, we
will further assume that $w=\Theta(\lg n)$.  In this sense, the cell probe model is a \emph{transdichotomous model}, as it provides a 
bridge between the problem size, $n$, and the cell or machine size, $w$.

\section{Dynamic Connectivity Lower Bound for Paths}

The lower bound we are trying to determine is the best achievable worst-cast time for a sequence of updates and queries to a path.  We 
will prove the following:

\begin{theorem}
Under the cell probe model, the lower bound worst-case cost is $\Omega(\lg n)$ per operation.
\end{theorem}

It is possible to show that the lower bound is also $\Omega(\lg n)$ in the amortized case, but we will only examine the worst case cost.

\subsection{Path Grid}

We start by arranging the $n$ vertices in a $\sqrt{n}$ by $\sqrt{n}$ grid, as shown in Figure 1.  The grid has perfect matching between 
consecutive columns $C_{i}$ and $C_{i+1}$.  This results in a graph with $\sqrt{n}$ disjoint paths across the columns.  The paths can 
be coded by the permutations $\pi_{1}, \pi_{2}, \ldots \pi_{\sqrt{n}-1}$, where $\pi_{i}$ is the permutation of edges that joins column 
$C_{i}$ and $C_{i+1}$.

\begin{figure}[h]
\begin{center}      
\leavevmode 
\scalebox{.5}{\includegraphics{grid.png}}     
\end{center}
\caption{A $\sqrt{n}$ by $\sqrt{n}$ grid of vertices, with one of the disjoint paths darkened.}
\end{figure}

\subsection{Block Operations}

There are two operations that we define on the grid:

\begin{itemize}
\item{\em UPDATE($i, \pi$)} -- Replaces the $i^{th}$ permutation $\pi_{i}$ in the grid with permutation $\pi$.
This is equivalent to $O(\sqrt{n})$ edge deletions and insertions.
\item{\em VERIFY\_SUM($i, \pi$)} -- Check if $\sum_{j=1}^i(\pi_{j}) = \pi$.  In other words, check if the first i permutations 
$\pi_{1}, \pi_{2}, \ldots \pi{i}$ is equivelent to a single permutation $\pi$.  This is equivalent to $O(\sqrt{n})$ connectivity queries.
\end{itemize}

We can now described the dynamic connectivity problem in terms of these operations and cell probe operations.  Thus, we make the following claim:

\begin{claim} \label{clm3}
Performing $\sqrt{n}$ \textsf{UPDATE($i, \pi$)} operations and $\sqrt{n}$ \textsf{VERIFY\_SUM($i, \pi$)} operations requires 
$\Omega(\sqrt{n} \sqrt{n} \lg n) = \Omega(n \lg n)$ cell probes.
\end{claim}

\subsection{Construction of Bad Access Sequences}

To prove the above claim, we need to use a family of random, 
``bad''
sequences so as to ensure that we achieve the true lower 
bound.  Otherwise, it would be possible for the data structure to tune to the input sequences, allowing it to take advantage of patterns 
in the sequences and run faster than it would in the worst case.  We want to come up with the most difficult sequence of operations and
sequences possible to ensure a worst case lower bound.  

First, we will alternate the update and query operations,  \textsf{UPDATE($i, \pi$)} and \textsf{VERIFY\_SUM($i, \pi$)}.  Then, we will 
carefully select the arguments to these operations as follows:

\begin{itemize}
\item {\em $\pi$ for UPDATE} -- The permutation $\pi$ for each \textsf{UPDATE} operation will be uniformly random, which will randomly change the result of each \textsf{SUM} operation (where \textsf{SUM} computes the sum of $i$ permutations, which must also be performed by  \textsf{VERIFY\_SUM}).
\item {\em $\pi$ for VERIFY\_SUM} -- Using uniformly random permutations is not appropriate, because it is easy for the operation to check 
that a permutation \textit{doesn't} match the sum; it only needs to find one path in the sum that doesn't satisfy $\pi$.  The worst case permutation is actually the ``correct'' permutation, $\pi = \sum_{j=1}^i(\pi_{j})$, because this forces the operation to check \textit{every} path before returning \textsf{TRUE}.
\item {\em $i$} -- For both operations, the $i$'s follow the \emph{bit reversal sequence}.  To generate this sequence, take all n-bit numbers $n_{i}$, write them in binary, then reverse (or ``mirror'') the bits in each number $n_{i}$ to create a new number $m_{i}$.  This results in a new sequence of numbers with some special properties.  For example, the sequence maximizes the \textsf{WILBER\_1} function, as it chooses a path with the maximal number of non-preferred children at each step.
\end{itemize}

\paragraph{Example} Using the argument selection rules above, we will alternate between update and query operations using worst case arguments, resulting in a ``bad'' access sequence.  For example, with 3-bit cells, the bit reversal sequence is
\[
(000_2, 100_2, 010_2, 011_2, 001_2, 101_2, 011_2, 111_2) = (0, 4, 2, 6, 1, 5, 3, 7)
\]
The access sequence we would give would then be
\[
QUERY(0, \pi_{correct} ), UPDATE(0, \pi_{random} ), QUERY(4, \pi_{correct} ), UPDATE(4, \pi_{random} ), \ldots
\]
Notice that the sequences of queries defined above interleaves betwen adjacent blocks perfectly.  

\subsection{Tree of Time}

To keep track of our position in the bit reversal sequence, we can store each number in the sequence in a leaf node of a perfect tree.  The resulting tree has the interesting property being ordered by time from left to right.  We refer to this tree as the \emph{tree of time}.  The tree of time for $w=3$ is shown in Figure 2.

\begin{figure}[h]
\begin{center}      
\leavevmode 
\scalebox{.5}{\includegraphics{timeTree.png}}     
\end{center}
\caption{The tree of time for $w=3$}
\end{figure}

Because we used the bit reversal sequence to order the keys in the tree of time, it has the additional property that the left and right subtrees of any non-leaf node interleave.  This interleaving is the reason that the bit reversal sequence is useful in our lower bound proof.  Each node in the right subtree needs to be aware of the \textsf{UPDATE} operations performed on the nodes in the left subtree.  In other words, the right subtree must read the writes made in the left subtree.  The resulting transfer of information between the interleaved left and right nodes is what makes the access sequence hard.

\begin{claim} \label{clm4}
For every node $v$ in the tree, let $\ell$ be the number of leaves in its subtree, corresponding to $\ell$ units of time.
Then the right subtree of $v$ is expected to execute $\Omega(l\sqrt{n})$ cell probes that read from cells last written during the left subtree.
\end{claim}

Because $\ell$ maps onto the columns $C_{0}, C_{1}, \ldots , C_{\sqrt{n}-1}$ of our grid of vertices, we know that $\ell = \sqrt{n}$ when $v$ is the root of the tree of time.  By summing over all levels of the tree, we end up summing over $\Omega(\lg n)$ levels of $v$; however, to avoid double-counting read-write pairs across subtrees, we must make sure to count the read of $r$ (in the right subtree) of the write $w$ (in the left subtree) only at $v = LCA(r, w)$.  Finally, using linearity of expectation gives us the bound from Claim \ref{clm3}:
\[
\Omega(l\sqrt{n}) \textrm{ per operation} \cdot \Omega(\lg n) \textrm{ operations} = \Omega(\sqrt{n} \sqrt{n} \lg n) = \Omega(n \lg n)
\]

\subsection{Proof Idea using Information Theory}

To prove Claim \ref{clm3}, we need to first prove Claim \ref{clm4}.  To accomplish this, we will assume that Claim \ref{clm4} is false, meaning that we assume we can beat $\Omega(l\sqrt{n})$ per cell probe and $\Omega(l \sqrt{n} \lg n)$ overall.  Showing that this assumption leads to a contradiction is sufficient to prove Claims \ref{clm4} and \ref{clm3}.  

To uncover the contradition we will use an encoding argument from information theory.  The idea is that there must be information transferred from the left subtree to the right tree, because every \textsf{UPDATE} in the left subtree must be known by the right subtree during its \textsf{VERIFY\_SUM} operations.  Under the cell probe model, information can only be transferred via cell reads.  Therefore, we need to determine a lower bound on the number of bits required to encode the permutations each read operation may need to recover.  Kolmogorov complexity provides us with the following theorem:

\begin{theorem} \label{thm6}
Encoding a single random permutation on $n$ elements requires $\Omega(n \lg n)$ bits.
\end{theorem}

If $v$ has $\ell$ leaves, then its left subtree has $\frac{\ell}{2}$ leaves corresponding to $\frac{\ell}{2}$ random permutations on $\sqrt{n}$ elements.  Applying Theorem \ref{thm6} yields the lower bound we are looking for.

\begin{lemma} \label{lem7}
Encoding $\frac{\ell}{2}$ random permutations on $\sqrt{n}$ elements requires $\Omega(\ell \sqrt{n} \lg n)$ bits.
\end{lemma}

\subsection{A Simpler Proof using \textsf{SUM}}

Our task is now to assume that Claim \ref{clm4} is false and show that this assumption leads to a contradiction with Lemma \ref{lem7}.  To do this, we start by assuming we know the entire history of permutations before $v$'s subtree.  Then, we encoded the verified sums in $v$'s right subtree such that we can recover the sequence of random permutations, $S(\pi_{i})$, from the left subtree.  If this encoding requires $\Omega(\ell \sqrt{n} \lg n)$ bits as decribed by Lemma \ref{lem7}, then we know this must be the lower bound for Claim \ref{clm4}.  However, this is a difficult proof, so we will start with a simpler case.  Instead of encoding the verified sums, we will instead encode the sums of the permutations.

\paragraph{Query:} 
\textsf{SUM($i$)}:  $\sum_{j=1}^i(\pi_{j})$, or the composition of the first $i$ permutations.

\paragraph{Setup:}
We know the entire past, including the permutations that occurred before $v$'s subtree.

\paragraph{Encoding:}
Let $R = \{\textrm{cells read during right subtree}\}$, and $W = \{\textrm{cells written during left subtree}\}$.  The set $P = R \cap W$ contains the cells read in the right subtree which were written to earlier, in the left subtree.  Now, we will try encoding $R \cap W$.  If the size of each cell is $w=\Theta(\lg n)$, then it will require $2w$ bits to encode addresses and contents of each cell in $R \cap W$.  The total encoding size is therefore $|R \cap W| \cdot \lg n$ bits.

\paragraph{Decoding:}
To decode, we first simulate all possible \textsf{SUM($i$)} queries in the right subtree.  This generates $R \cap W$.  Decoding now depends on where the cell was last written:
\begin{itemize}
\item {\em Right subtree} -- If the cell was last written in the right subtree, we have access to the permutations performed, and decoding is trivial.
\item {\em Left subtree} -- We can use the information encoded in the simulated $R \cap W$ set to determine what permutations were performed by the left subtree's \textsf{UPDATE} operations.
\item {\em Past subtrees} -- Because we assume we know the entire past, we know the permutations caused by past subtrees.
\end{itemize}

\paragraph{Result:}
We know by Lemma \ref{lem7} that our encoding must be at least $\Omega(\ell \sqrt{n} \lg n)$ bits.  This maps onto to set we are encoding, $|R \cap W| \cdot \lg n$.  By cancelling out the $\lg n$ terms, we discover that $|R \cap W| = \Omega(\ell \sqrt{n})$, which agrees with Claim \ref{clm4}.

\subsection{Proof using \textsf{VERIFY\_SUM}}

We have shown that Claim \ref{clm4}'s bound of $\Omega(l\sqrt{n})$ cell probes is necessary to transfer the information from the \textsf{UPDATE} operations on the left subtree to the \textsf{SUM} operations on the right.  Now we will try to prove a similar result using \textsf{VERIFY\_SUM}.  

\paragraph{Query:} 
\textsf{VERIFY\_SUM($i, \pi$)}:  $\sum_{j=1}^i(\pi_{j}) =? \pi$.  Returns \textsf{TRUE} if the composition of the first $i$ permutations is equivalent to permutation $\pi$, and returns \textsf{FALSE} otherwise.  

\paragraph{Setup:}
Again, we assume we know the entire past.  We also know that, for any $i$, only one $\pi$ will result in \textsf{TRUE}.  Furthermore, recall that we are using the worst case $\pi$ inputs for \textsf{VERIFY\_SUM} operations.  Because the worst case $\pi$ is the ``correct'' permutation, we know that \textsf{VERIFY\_SUM} will always return true!

\paragraph{Encoding:}
As before, define $R$, $W$, and $R \cap W$.  Instead of just having to encode the sums, however, we also have to encode the input $\pi$'s.  This is avoided by noting that the input $\pi$ must always be the permutation that causes \textsf{VERIFY\_SUM} to return true, as described in the setup.  Instead of encoding $\pi$, we can just recover during decoding by trying all possible $\pi_{i}$ permutations and selecting the one which matches the encoded $R \cap W$ permutation.

\paragraph{Decoding:}
We start by simulating all possible input permutations to \textsf{VERIFY\_SUM} so as to recover $\pi$, as described above.  As before, the decoding algorithm relies on the knowledge of where the cell was last written.  Unfortunately, this is no longer easy to discern.  Because we simulated all possible input permutations, we queried cells in set $R'$ as well as in $R$.  Let $R'$ be the set of cells read by these incorrect queries.  If we read cell $r \in R'$, $r \notin R$, then the permutation $\pi$ must be incorrect.  However, there is a chance that $r$ will intersect
with $W$ in cells not in $R \cap W$, so that the state data needed to evaluate these read operations will get the value from the past, instead of  getting data written during the left subtree.  Since the algorithm is reading incorrect bits on its cell probes when the query has an answer that should be no, it might return yes instead, resulting in a false positive.

\paragraph{Result:}
We could get around this problem by encoding all of $R$ or $W$, allowing us to check whether $r \in W \setminus R$ or 
$r \in \textrm{past} \setminus (R \cap W)$.  However, this requires a large number of bits; $W$ could be as large as
$\ell \sqrt{n} \log n$ cells, or $\ell \sqrt{n} \log^2 n$ bits, prevent our encoding from fitting in the desired $\ell \sqrt{n} \log n$ space.  
Instead, we will try to find a way to cheaply encode whether we are in $R \setminus W$ or $W \setminus R$.

\begin{figure}[h]
\begin{center}      
\leavevmode 
\scalebox{.5}{\includegraphics{venn.png}}     
\end{center}
\caption{Venn diagram showing the intersections of $R$, $W$, and $R'$.}
\end{figure}

\subsection{Separators}

To solve the decoding problem described for \textsf{VERIFY\_SUM} queries, we encode a \emph{separator}.

\begin{definition}
A \emph{separator family} for size $m$ sets is a family $S \subset
2^U$ with the property that for any $A, B \subset U, |A|, |B| \le m$,
there exists $C \in S$ such that $A \subset C, B \subset U \setminus
C$.
\end{definition}

\begin{theorem} \label{thm8}
There exist separator families $S$ such that $|S| \le 2^{O(m + \log
\log U)}$. 
\end{theorem}

Theorem \ref{thm8} states that we can encode a separator $S$ in $O(|R| + |W| + \lg \lg n)$ bits.  The separator 
will have the property that $R \setminus W \subseteq S$ and $W \setminus R \subseteq \overline{S}$.  By encoding
the separator $S$ along with $R \cap W$, we can successfully decode each write in the following manner. 

\paragraph{Decoding (with Separator):}
To decode, we first simulate all input permutations to \textsf{VERIFY\_SUM} so as to recover $\pi$.  Then, we  determine when the cell being read was last written:
\begin{itemize}
\item {\em Right subtree} -- As before, we have knowledge of the permutations performed on the right subtree, so decoding here is trivial.
\item {\em $R \cap W$} -- We again use the information encoded in the simulated $R \cap W$ set to determine what permutations were performed.
\item {\em $S$} -- If the cell is in $S$ but not $R \setminus W$, then it must have last been written in the past, and we assume we know the effects of past permutations
\item {\em $\overline{S}$} -- If the cell is not in $S$, then it must not be in $R$, meaning that this can't be the correct $\pi$.  So we abort this decoding and look elsewhere for the appropriate $\pi$.
\end{itemize}

\begin{figure}[h]
\begin{center}      
\leavevmode 
\scalebox{.5}{\includegraphics{separator.png}}     
\end{center}
\caption{The separator $S$ distinguishing between $R \setminus W$ and $W \setminus R$.}
\end{figure}

\subsection{Conclusion}

We have shown that we can decode the permutations caused by the left subtree \textsf{UPDATE} operations by using an encoded 
representation of $R \cap W$ and a separator $S$ of size $O(|R| + |W| + \lg \lg n)$, giving us a lower bound of 
\[
|R \cap W| O(\lg n) + O(|R|+|W| + \lg\lg n) = \Omega(\ell \sqrt{n} \lg n).
\]
There are two possibilities to consider.
\begin{itemize}
\item {\em $|R| + |W| = \Omega(\ell \sqrt{n} \lg n)$} -- If $|R| + |W|$ is large, then we have not proven anything, because our estimate is
then trivially true.  However, in this case, there were a total of
$O(\ell)$ operations handled by each of the left and right subtrees,
each of which does $\sqrt{n}$ dynamic connectivity operations.  Then
at least one of those dynamic connectivity operations used
$\Omega(\lg n)$ time, and the main theorem holds.  
\item {\em $|R| + |W| = o(\ell \sqrt{n} \lg n)$} -- This
implies that there were at least that many cell probes were forced on the
right subtree by the left subtree, which is exactly what we were
trying to show.  
\end{itemize}

%\bibliography{mybib}
\bibliographystyle{alpha}

\begin{thebibliography}{77}

\bibitem{old1}
M. L. Fredman and M. R. Henzinger. 
\emph{Lower bounds for fully dynamic connectivity problems in graphs}.
Algorithmica, 22(3):351–362, 1998.

\bibitem{old2}
P. B. Miltersen, S. Subramanian, J. S. Vitter, and R. Tamassia. 
\emph{Complexity models for incremental computation}. 
Theoretical Computer Science,
130(1):203–236, 1994.

\bibitem{euler}
M. R. Henzinger, V. King, 
\emph{Randomized dynamic graph algorithms with polylogarithmic time per operation}. 
STOC 1995: 519-527

\bibitem{dyn}
J. Holm, K. Lichtenberg, M. Thorup, 
\emph{Poly-logarithmic deterministic fullydynamic algorithms for connectivity, minimum spanning tree, 2-edge, and biconnectivity}.  
J. ACM 2001: 48(4): 723-760

\bibitem{ccp}
M. Fredman, M. Saks, 
\emph{The cell probe complexity of dynamic data structures}. 
STOC 1989: 345-354

\bibitem{dc}
M. P\v{a}tra\c{s}cu, E. D. Demaine, 
\emph{Lower bounds for dynamic connectivity}. 
STOC 2004: 546-553


\end{thebibliography}

\end{document}