"Perl script is getting stuck for no reason"

There's always a reason. An apparent lack of reason is an indication of inadequate investigation.

"...it is getting stuck at random..."

That is almost impossible; computers are useful because they are deterministic. Of course you're probably aware that computers don't behave randomly, or you wouldn't be asking for assistance with the behavior you're seeing. But the sooner you abolish all notion that something is happening randomly, the sooner you'll be on your way to diagnosing the problem yourself.

The code you posted doesn't compile. But you're not asking about why it doesn't compile, you're asking why your code seems to hang. So that tells me that the code you provided for us to look at is not exactly the code you're running. That makes it difficult for us to know what's wrong.

Fixing the right curly bracket, I then get three warnings. Have you investigated why you're getting warnings? Are you getting warnings, or is that something that only happens in the code you provided us?

We can't debug this for you; we don't have the data, or the real code. But let me offer these suggestions, which with a little diligence, will probably give you an answer. For what it's worth, these are steps I would probably take.

• Add "warn" statements that tell you where you are within your code, and where you are within loops.
• Run this with "top" going in another terminal. Watch for memory growth. The "warn" statements previously mentioned will help you to see what parts of your script are consuming memory, as you follow the runtime with top.
• Test with a much smaller subset of your data sets. This will allow for quicker iterations of the debug/test cycle.
• Even with your much smaller subset, don't be satisfied with any substantial growth in memory usage as the script progresses. Keep in mind that you are dealing with very large files. If it becomes too costly to hold intermediate data in memory, put intermediate results into files or a database, and process those files rather than trying to hold the entire computation in memory at once.

If it seems like I'm talking a lot about memory, it's because without a running script and small test data set, I can only go with instinct, and my instinct is that when I see some data structures persisting throughout your script, and read that you're dealing with 800MB, 16GB, and 500MB files, and when I read that your complaint is the script grinding to a halt, you are consuming too much memory and bringing your system to its knees.

---

---

Posts are HTML formatted. Put <p> </p> tags around your paragraphs. Put <code> </code> tags around your code and data!

Titles consisting of a single word are discouraged, and in most cases are disallowed outright.

Read Where should I post X? if you're not absolutely sure you're posting in the right place.

Please read these before you post! —

• How do I compose an effective node title?
• How do I post a question effectively?
• Markup in the Monastery

Posts may use any of the Perl Monks Approved HTML tags:

a, abbr, b, big, blockquote, br, caption, center, col, colgroup, dd, del, details, div, dl, dt, em, font, h1, h2, h3, h4, h5, h6, hr, i, ins, li, ol, p, pre, readmore, small, span, spoiler, strike, strong, sub, summary, sup, table, tbody, td, tfoot, th, thead, tr, tt, u, ul, wbr

You may need to use entities for some characters, as follows. (Exception: Within code tags, you can put the characters literally.)

	For:		Use:
	&		&
	<		&lt;
	>		&gt;
	[		&#91;
	]		&#93;

Link using PerlMonks shortcuts! What shortcuts can I use for linking?

See Writeup Formatting Tips and other pages linked from there for more info.