"The PRE tag:"

I guess you're aware that <code>...</code> and <c>...</c> are preferred. These have the benefits of: not requiring entities (e.g. you don't need to change & to &amp;, and so on); wrapping long lines; and, in block form, supplying a [download] link to the original text. As you saw with your "Hebrew letters", entities may be rendered (e.g. ם becomes &#1501;). I tend to use <code> for blocks and <c> for inline text; however, they are interchangeable.

You get none of those benefits with <pre>...</pre>: you'll need to change special characters to entities manually (e.g. > to &gt;); you should also aim to keep lines short to avoid messing up the overall page view. What <pre> is particularly useful for is rendering characters outside the 7-bit ASCII range; <code> will render some of those (e.g. é to é and © to ©) but not others (e.g. ē to &#275; and ☺ to &#9786;) — I'm never going to remember the full list of which render as characters or entities, so I just take the pragmatic approach as use <pre> for all characters outside the 7-bit ASCII range. For inline work, I find <tt>...</tt> useful.

"To me, this whole site seems like a giant maze. I'm still figuring things out."

Yes, there's a lot of information and it can be daunting. Here's three nodes that I do recommend you at least know about and bookmark for future reference: "Writeup Formatting Tips", "Markup in the Monastery" and "Perl Monks Approved HTML tags". Also note that PerlMonks' HTML is not HTML5: believing they're the same is a common mistake many people make.