It looks like this has been solved before:
If you only need integers you can use BigInteger.js.
While looking for an big integer library for an ElGamal crypto implementation I tested several libraries with the following results:
I recommend this one: Tom Wu's jsbn.js (http://www-cs-students.stanford.edu/~tjw/jsbn/)
- Comprehensive set of functions and fast
Leemon Baird's big integer library (http://www.leemon.com/crypto/BigInt.js)
- Comprehensive set of functions and pretty fast
- BUT: Negative number representation is buggy!
- Pretty complete set of functions
- BUT: Converting really big numbers from strings into BigNumber objects result in INFINITY
- JS-Implementation of Scheme arithmetic functions
- BUT: No function for y= x^e mod n
I haven't tested this by myself: BigNumber (http://jsfromhell.com/classes/bignumber)
- Functions for high precision claculations
- BUT: It's said to be slow due to internal representation of numbers as strings
(Note that I haven't used this myself. Try it and see what you think.)