Hi,
Depends whether if you are opening (reading the file) or passing it as it is without reading.
In my previous project,
1) where we were reading the file using content conversion, the max size used to be around 5 mb.(In this case if you want to read bigger files , please go for recordset per messges,where you can specify how many records could be present in oe message)
2)Also, for passthrough interfaces where we were just passing the file using ICO, we could transfer files of size 20mb.(Here for even bigger files you can break the file into pieces and then combine them at receiver file channel)
But , basically it depends on the hardware configurations.There are many links for similar questions.
http://scn.sap.com/thread/1021753