Fixed version with a quick demonstration of the problem: 

#include <stdio.h>
#include <stdint.h>
#include <stdlib.h>

uint64_t gcm2( uint64_t max, uint64_t lcm ) {

    int b = __builtin_ctzl( lcm & 0xfff );
    lcm <<= ( 12 - b );
    return ( max / lcm ) * lcm;
}

uint64_t gcm(uint64_t max, uint64_t lcm)
{
    lcm <<= 12 - __builtin_ctzl(lcm | 0x1000);
    return max - (max % lcm);
}


int main(int argc, char *argv[])
{
    unsigned long max = 12345 << 10;
    while (argc-- > 1) {
        unsigned long v = strtoul(argv[argc], NULL, 10);

        printf("gcm2(%lu,%lu) = %lu\n", max, v, gcm2(max, v));
        printf("gcm(%lu,%lu) = %lu\n", max, v, gcm(max, v));
    }
    return 0;
}

[download]
$ ./a.out 12 77 4096
gcm2(12641280,4096) = 0
gcm(12641280,4096) = 12640256
gcm2(12641280,77) = 12615680
gcm(12641280,77) = 12615680
gcm2(12641280,12) = 12632064
gcm(12641280,12) = 12632064

In reply to Re^4: Simple arithmetic? (And the winner is ... ) by oiskuu
in thread Simple arithmetic? by BrowserUk

Title:
Use:  <p> text here (a paragraph) </p>
and:  <code> code here </code>
to format your post, it's "PerlMonks-approved HTML":


  • Posts are HTML formatted. Put <p> </p> tags around your paragraphs. Put <code> </code> tags around your code and data!
  • Titles consisting of a single word are discouraged, and in most cases are disallowed outright.
  • Read Where should I post X? if you're not absolutely sure you're posting in the right place.
  • Please read these before you post! —
  • Posts may use any of the Perl Monks Approved HTML tags:
    a, abbr, b, big, blockquote, br, caption, center, col, colgroup, dd, del, details, div, dl, dt, em, font, h1, h2, h3, h4, h5, h6, hr, i, ins, li, ol, p, pre, readmore, small, span, spoiler, strike, strong, sub, summary, sup, table, tbody, td, tfoot, th, thead, tr, tt, u, ul, wbr
  • You may need to use entities for some characters, as follows. (Exception: Within code tags, you can put the characters literally.)
            For:     Use:
    & &amp;
    < &lt;
    > &gt;
    [ &#91;
    ] &#93;
  • Link using PerlMonks shortcuts! What shortcuts can I use for linking?
  • See Writeup Formatting Tips and other pages linked from there for more info.