If you get away with "probably unique", you can rely on statistics: by randomly selecting with replacement a small number of elements from a very large set, you can determine the likelihood that all are unique (something like (k-1)!/n^k for k pulls from a set of n).

If you can't get away with "probably", you need to use something sequential (1, 2, 3, ...) or some invertible function of something sequential (f(1), f(2), f(3)...), if you are concerned that these "sequential" numbers don't appear to be sequential. (security-thru-obscurity, beware)

Otherwise, you're stuck with remembering what you've already said: you need a database.

An aside -- there are special approaches for storing "archival" data, where one only needs insert and select, not delete or update. Here's one random link to this idea.

In reply to Re: Maintaining uniqueness across millions of items by water
in thread Maintaining uniqueness across millions of items by sdbarker

Title:
Use:  <p> text here (a paragraph) </p>
and:  <code> code here </code>
to format your post, it's "PerlMonks-approved HTML":


  • Posts are HTML formatted. Put <p> </p> tags around your paragraphs. Put <code> </code> tags around your code and data!
  • Titles consisting of a single word are discouraged, and in most cases are disallowed outright.
  • Read Where should I post X? if you're not absolutely sure you're posting in the right place.
  • Please read these before you post! —
  • Posts may use any of the Perl Monks Approved HTML tags:
    a, abbr, b, big, blockquote, br, caption, center, col, colgroup, dd, del, details, div, dl, dt, em, font, h1, h2, h3, h4, h5, h6, hr, i, ins, li, ol, p, pre, readmore, small, span, spoiler, strike, strong, sub, summary, sup, table, tbody, td, tfoot, th, thead, tr, tt, u, ul, wbr
  • You may need to use entities for some characters, as follows. (Exception: Within code tags, you can put the characters literally.)
            For:     Use:
    & &amp;
    < &lt;
    > &gt;
    [ &#91;
    ] &#93;
  • Link using PerlMonks shortcuts! What shortcuts can I use for linking?
  • See Writeup Formatting Tips and other pages linked from there for more info.