\documentclass[11pt]{article}
\usepackage{latexsym}
\usepackage{amsmath}
\usepackage{amssymb}
\usepackage{amsthm}
\usepackage{epsfig}
\usepackage{psfig}

\newcommand{\handout}[5]{
  \noindent
  \begin{center}
  \framebox{
    \vbox{
      \hbox to 5.78in { {\bf 6.897: Advanced Data Structures } \hfill #2 }
      \vspace{4mm}
      \hbox to 5.78in { {\Large \hfill #5  \hfill} }
      \vspace{2mm}
      \hbox to 5.78in { {\em #3 \hfill #4} }
    }
  }
  \end{center}
  \vspace*{4mm}
}

\newcommand{\lecture}[4]{\handout{#1}{#2}{#3}{Scribe: #4}{Lecture #1}}

\newtheorem{theorem}{Theorem}
\newtheorem{corollary}[theorem]{Corollary}
\newtheorem{lemma}[theorem]{Lemma}
\newtheorem{observation}[theorem]{Observation}
\newtheorem{proposition}[theorem]{Proposition}
\newtheorem{definition}[theorem]{Definition}
\newtheorem{claim}[theorem]{Claim}
\newtheorem{fact}[theorem]{Fact}
\newtheorem{assumption}[theorem]{Assumption}

% 1-inch margins, from fullpage.sty by H.Partl, Version 2, Dec. 15, 1988.
\topmargin 0pt
\advance \topmargin by -\headheight
\advance \topmargin by -\headsep
\textheight 8.9in
\oddsidemargin 0pt
\evensidemargin \oddsidemargin
\marginparwidth 0.5in
\textwidth 6.5in

\parindent 0in
\parskip 1.5ex
%\renewcommand{\baselinestretch}{1.25}

\begin{document}

\lecture{14 --- March 31, 2005}{Spring 2005}{Prof.\ Erik Demaine}{Jelani Nelson}

\section{Overview}
In the last lecture, we discussed algorithms for integer sorting.  In this
lecture we will discuss the relationship between integer sorting algorithms
and priority queue data structures.  It is obvious that fast priority
queues imply fast sorting, since if a priority queue can do \textsc{Insert}
and \textsc{Delete-min} in $O(f(n))$, then we can sort in $O(nf(n))$ by
inserting all elements into the queue then calling \textsc{Delete-min} $n$
times.  So, we will focus in this lecture on the reverse --- that is, how
to make fast priority queues once we have fast sorting algorithms.


\section{Priority Queue}
A priority queue, also sometimes called a {\em heap}, is a data structure
that performs the following operations:
\begin{itemize}
\item \textsc{Insert}($x,key$) -- Insert the element $x$ into the heap
  with key $key$.

\item \textsc{Delete-Min}() -- Return the smallest element in the heap
  under key order.  Also, delete this element from the heap.

\item \textsc{Decrease-Key}($x,key$) -- Change the key of item $x$ in
  the heap to $key$.  $key$ must not be greater than $x$'s current key
  value.

\item \textsc{Meld}($H_1, H_2$) -- Takes two heaps as input and
  produces a new heap that contains all the elements of the two that
  are input.
\end{itemize}

The fundamental operations are \textsc{Insert} and
\textsc{Delete-Min}. The other two operations are motivated by
applications to graph algorithms. 

Two old but very important algorithms in computer science that require
heaps are Dijkstra's shortest path algorithm in \cite{dijkstra}, and
Prim's algorithm for minimum spanning tree, discussed in \cite{prim}.
Note one difference in the types of priority queues needed in the two
algorithms: in Dijkstra's algorithm the minimum key in the heap never
decreases.  We call heaps that only support such situations {\em
monotone}.  If we remember the way Dijkstra's algorithms and Prim's
algorithm work, each of them performs $O(m)$ decrease-key operations,
and $O(n)$ \textsc{Insert}'s and \textsc{Delete}'s. This motivates the
need for \textsc{Decrease-Key} being faster than \textsc{Delete-Min}.

Another old algorithm that uses the priority queue is Edmonds'
algorithm for directed MST \cite{edmonds}. This can be implemented
efficiently using meldable priority queues.

\subsection{Results}
For the following results, you can assume that a fast \textsc{Meld} is
not supported unless otherwise stated.  In the comparison model, a
normal binary heap can achieve $O(\log n)$ per operation.  Fibonacci
heaps, introduced in \cite{fibheap}, achieve $O(1)$
\textsc{Decrease-Key} and \text{Insert} and $O(\log n)$
\textsc{Delete-Min} (all times amortized).  Note, this implies an
$O(n\log n+m)$ running time for Prim's and Dijkstra's algorithms. A
worst-case version of the Fibonacci heap, which also achieves $O(1)$
inserts, was shown in \cite{brodal}.

For integer priority queues, the atomic heap was introduced in
\cite{atomic}, which achieves $O(1)$ amortized time per operation for
$O(\hbox{polylog}n)$ elements.  From a high-level point of view, the
atomic heap works much like a fusion tree (and thus even supports
predecessor and successor queries).  The paper discusses how to use
this heap for an optimal $O(n+m)$ MST algorithm (note that this is not
through direct implementation of any classic algorithm).  The only
linear time algorithm for MST known in the comparison model is
randomized, found in \cite{karger}.  The best deterministic MST
algorithm, due to \cite{chazelle}, runs in time $O((n+m)\alpha(n))$.

As far as relating sorting to priority queues, it was shown in
\cite{thorup00} that $O(n \cdot S(n,w))$ sorting implies an
$O(S(n,w))$ amortized per operation monotone priority queue.  This
result was later improved to worst-case and without the monotonicity
condition in \cite{thorup02}.  Work has also been done in showing how
to use a fast priority queue to get a fast meldable priority queue.
It is shown in \cite{meld} that an $O(P(n,w))$ time per operation
priority queue implies an $O(P(n,w)\alpha(n))$ time per operation
meldable priority queue.  The $\alpha(n)$ was essentially removed
(disappears for the $P(n,w)$ we currently know and care about) in
\cite{bettermeld}.

The best known priority queue with constant \textsc{Decrease-Key} does
not match the current sorting bound. Rather, it achieves $O(\lg\lg n)$
per \textsc{Delete-Min} \cite{thorup03}. Beside MST, another important
application of such queues was solved through different techniques:
single-source shortest paths were shown to be computable in linear
time, $O(m+n)$, in \cite{thorup99}. However, this is only for the
undirected case, and the directed version still relies of priority
queues with constant \textsc{Decrease-Key}.

The rest of this lecture discusses what appears to be a simpler way to
convert $O(n \cdot S(n,w))$ sorting to priority queues with operations
supported in $O(S(n,w))$ time. This is an ad-hoc creation of your
teachning staff and has not been checked extensively, so it should be
trusted accordingly.


\subsubsection{The Priority Queue}
We will keep $\Theta(\lg n)$ buckets, $B_0,B_1,...$.  Each $B_i$ will
either be:

\begin{itemize}
\item \underline{Empty}
\item \underline{Sorted} and of size at most $2^i$, stored in a linked list
(we will charge to pay for the sort).
\item \underline{Semi-structured}:
   \begin{itemize}
   \item[-] Unstructured part $U_i$ of arbitrary size, stored in a linked
   list (we will be able to charge the elements of $U_i$).
   \item[-] Structured part $S_i$ split into $\Omega(2^{i/2})$ groups of
   size $\Theta(2^{i/2})$ each.  Each group is unsorted, but the groups are
   in order.  In our charging argument, we will charge for the groups
   beyond $\Theta(2^{i/2})$.
   \end{itemize}
\end{itemize}

We will always keep $B_0$ semi-structured.  For semi-structured
buckets $B_i,B_j$, if $i<j$ then all elements of $B_i$ will be less
than all elements of $B_j$.  For sorted $B_i$, we will maintain the
invariant that min$(B_{i-1}) < $max$(B_i) < $min$(B_{i+1})$.

We will keep an atomic heap on min$(B_i)$ for all non-empty $B_i$ to find
which bucket contains the element with minimum key.  Recall that atomic
heaps have $O(1)$ time per operation on $O(\hbox{polylog}n)$ elements, and
here there are $\Theta(\log n)$ buckets.  \textsc{Decrease-Key} will remove
from the element from the existing bucket then call \textsc{Insert} again.

For \textsc{Delete-Min}:
\begin{itemize}
\item[-] Find the lowest non-empty bucket $B_i$.
\item[-] if $B_i$ is sorted, simply extract its minimum from the linked
list.
\item[-] if $B_i$ is semi-structured:
   \begin{itemize}
   \item[-] if $|B_i| = O(2^i)$, then sort everything and turn it into a
   sorted bucket.
   \item[-] if $|B_i| = \omega(2^i)$, then we do an ``exponential blastoff''
      \begin{itemize}
      \item[-] for $j=i,i+1,...$
         \begin{itemize}
         \item[-] Put elements of $B_j$ into a bag.
         \item[-] Drop the smallest $\Theta(2^j)$ elements from the bag
         into $B_j$ (we figure out which are smallest by going through the
         elements a first time and doing recursive medians on the set).
         \item[-] Stop when the bag is empty.
         \end{itemize}
      \end{itemize}
   \end{itemize}
\end{itemize}

Note that the recursive medians on the set above runs in linear
time.  This is because the sizes of the $B_j$ increase as a
geometric series.  So, our time for the recursive medians is the sum
of a geometric series multiplied by $O(n)$, which is just $O(n)$. It
was shown in \cite{hanthorup} that structured parts can be built
from unstructured parts in $O(2^i)$ time.  Also, we can split and
merge structured parts in $O(2^{i/2})$ time.

%\bibliography{mybib}
\bibliographystyle{alpha}

\begin{thebibliography}{77}
\bibitem{dijkstra}
E.W. Dijkstra. A note on two problems in connection with graphs. In
Numerische Mathematik, pages 1:269-271, 1959.

\bibitem{prim}
R.C. Prim. Shortest connection networks and some generalizations. Bell
Sys. Tech. Journal, pages 1389--1401, 1957.

\bibitem{edmonds}
J. Edmonds, ``Optimum branchings'', J. Research of the National Bureau of
Standards, 71B, 1967, pp.233-240.

\bibitem{fibheap}
Michael L. Fredman, Robert Endre Tarjan: Fibonacci heaps and their uses in improved network optimization algorithms. J. ACM 34(3): 596-615 (1987)

\bibitem{brodal}
Gerth Stølting Brodal: Worst-Case Efficient Priority Queues. SODA 1996:
52-58

\bibitem{atomic}
Michael L. Fredman, Dan E. Willard: Trans-Dichotomous Algorithms for
Minimum Spanning Trees and Shortest Paths. J. Comput. Syst. Sci. 48(3):
533-551 (1994)

\bibitem{karger}
David R. Karger, Philip N. Klein, Robert Endre Tarjan: A Randomized
Linear-Time Algorithm to Find Minimum Spanning Trees. J. ACM 42(2): 321-328
(1995)

\bibitem{chazelle}
Bernard Chazelle: A minimum spanning tree algorithm with Inverse-Ackermann
type complexity. J. ACM 47(6): 1028-1047 (2000)

\bibitem{thorup99}
Mikkel Thorup: Undirected Single-Source Shortest Paths with Positive
Integer Weights in Linear Time. J. ACM 46(3): 362-394 (1999)

\bibitem{thorup00}
Mikkel Thorup: On RAM Priority Queues. SIAM J. Comput. 30(1): 86-109 (2000)

\bibitem{thorup02}
Mikkel Thorup: Equivalence between Priority Queues and Sorting. FOCS 2002: 125-134

\bibitem{thorup03} 
Mikkel Thorup: Integer priority queues with decrease key in constant
time and the single source shortest paths problem. STOC 2003: 149-158

\bibitem{meld}
Ran Mendelson, Mikkel Thorup, Uri Zwick: Meldable RAM priority queues and
minimum directed spanning trees. SODA 2004: 40-48

\bibitem{bettermeld}
Ran Mendelson, Robert Endre Tarjan, Mikkel Thorup, Uri Zwick: Melding
Priority Queues. SWAT 2004: 223-235

\bibitem{hanthorup}
Yijie Han, Mikkel Thorup: Integer Sorting in 0(n sqrt (log log n)) Expected Time and Linear Space. FOCS 2002: 135-144

\end{thebibliography}
\end{document}
