One thing I noticed though was that adding network support seems tricky

Yes, indeed. Programming 8 bit AVRs (ATtiny, ATmega, ATxmega) is comparable to programming the old home computers, but in C (and perhaps some Assembler) instead of BASIC (and almost always Assembler). Your code runs on the bare hardware, there is no operating system, you have to implement everything by yourself (or find good libraries).

Networking can be easy, for some kind of networks. Many AVRs have hardware support for I²C (called two-wire interface), SPI, the "better" ones also have a hardware UART capable of implementing RS232 and RS485 (except for the line level inverters), the "big" ones also have a CAN bus controller.

But the UART lacks a FIFO that you are used to from "big" computers, so you need a fast interrupt routine to buffer incoming data and another one to send buffered outgoing data. This is also true for I²C, SPI, and probably even CAN. And due to the low clock frequency (typically much less than 50 MHz), your code runs slow compared to any PC you might use today, even with a highly optimizing compiler and a RISC instruction set. You simply won't be able to handle data coming in at Gigabit Ethernet speeds, no matter what interface you use. Probably not even at 10 MBit/s. And you don't have the memory to do so, not even enough address lines.

So, what does work?

• As written above, I²C, SPI, RS232, RS485, CAN to communicate with other microcontrollers or a network gateway.
• You can use old ISA Ethernet cards (10 MBit/s), and implement a low level ethernet driver and some kind of minimal TCP/IP (IPX, whatever) on top of that. There are libraries out there, so you don't have to implement all by yourself. Don't expect to be able to handle full TCP/IP, and don't expect to be able to handle it at full speed.
• There is at least one ethernet chip (ENC28J60, 10 MBit/s I guess) that can be controlled via SPI and needs just a few passive components, so you don't have to implement a minimal ISA bus. The chip can deliver a clock signal to the AVR, and it also has a Wake-on-LAN signal output. So, you need fewer pins than for the ISA solution, but on the software side, there is little difference. You need a low level driver and a TCP/IP or IPX stack, and it won't be fancy.
• If you can use RS232, either bit-banged or using a hardware UART, you can use one of the old modem protocols for file uploads / downloads (XModem, YModem, ZModem). Or you implement PPP and get TCP/IP. You "just" need some gateway that routes your PPP connection to your ethernet. Old cheap routers flashed with a custom firmware (OpenWRT) can do that. Plus, most of them have a CMOS/TTL level RS232 interface, so you can omit the line level inverters. A raspi could do that, too.
• Even the smallest ATtiny can implement a low-speed USB device completely in software (plus three resistors, two zener diodes, one electrolyte capacitor, one ceramic capacitor, no crystal). The V-USB library consumes almost all of its resources, but leaves enough room for a little bit of I/O, powered by the USB host. You can implement USB HID devices (fake a keyboard, mouse, joystick), USB CDC ACM (RS232 emulation), and perhaps other devices. That's also my current pet project (based on CDC-IO), an ATtiny85 connecting a Linux USB Host to three solid state relais to switch peripherals on and off. Linux sees a low speed USB CDC ACM device (USB spec says those devices must not exist, CDC ACM is reserved for full speed and faster), just writing to /dev/ttyACM0 is sufficient to control the relais. No one prevents you from talking PPP over USB CDC ACM. It will be slow, due to the USB bit-banging overhead.
• There is also USB CDC Ethernet, simulating two Ethernet cards and a cross-over cable. Never tested that with V-USB, and no clue how to implement that. And there is Microsoft's RNDIS, NDIS over USB. Never tested that, no clue how to implement it. But in both cases, you need a router to get into the "real" ethernet. As above, this could be a raspi or an OpenWRT box with a USB host controller.

Note that the really big AVRs contain USB controller hardware, so you don't need bit-banging to communicate with the USB host. But those AVRs come only in SMD packages.

AVRs are cheap. Consider using two AVRs, one for the real-time task and one for the communication with the outer world. Use I²C, SPI or similar for a simple, fast protocol in between those two AVRs. Or use I²C, SPI, or RS232 between an AVR and a raspi.

Alexander