Best case, the startup is really successful and you help build a team of people and an agile process that works for your company.
Worst case scenario, you do it for a year or two and then walk into another job with 1-2 years experience and get a really good salary.
Just the fact that you are aware of agile approaches puts you at an advantage. Find a process that works for you as one developer and then as more people come on board use their experience to adjust it so that it works for more than one person.
There isn't one agile approach that just works, everyone modifies what they are doing to make it work for their team and their company.
It will look good on your CV. As the only developer/technical person you are going to get a huge amount of experience.
My first job after uni was in a very small 4 man company, it didn't do me any harm. Agile hadn't even been invented when I started work... so get off my lawn!