We still don't know why Luke Skywalker is on that lonesome island on a lonesome planet at the end of Star Wars: The Force Awakens. Presumably we'll learn a lot more when Star Wars: The Last Jedi comes out. But don't expect a thorough history of everything that happened to the (ex?) Jedi since the credits rolled on Return of the Jedi - even if all of that does exist in the mind of Mark Hamill and director Rian Johnson.
“There’s massive amounts of backstory that is left to your imagination and I couldn’t do my job without figuring out what that was,” Hamill explained in an interview with Entertainment Weekly. “Since it’s not really important to the main story as a whole a lot of it is just for my own process. I talked with Rian about it and went into this elaborate scenario of what happened to Luke after the end of the Return of the Jedi.”
Johnson said that the first step in writing The Last Jedi was "figuring out why [Luke's] on that island". All we know is that things went very wrong with his new Jedi Order, and it sounds like Johnson had a lot of freedom to take the story wherever he wanted beyond that.
“We know that he is not a coward," Rian said. "He’s not just hiding because he’s scared. But we also know that he must know his friends are in danger. He must know the galaxy needs him. And he’s sitting on this island in the middle of nowhere. There had to be an answer. It had to be something where Luke Skywalker believes he’s doing the right thing - and the process of figuring out what that is and unpacking it is the journey for Rey.”
I mean, if I was accidentally responsible the rise of a new Dark-Side prodigy who seems set to repeat the endless cycle of galactic suffering, I'd probably get a bit withdrawn too.
“[Luke] made a huge mistake in thinking that his nephew was the chosen one, so he invested everything he had in Kylo, much like Obi-Wan did with my character,” Hamill said. “And he is betrayed, with tragic consequences. Luke feels responsible for that. That’s the primary obstacle he has to rejoining the world and his place in the Jedi hierarchy, you know? It’s that guilt, that feeling that it’s his fault, that he didn’t detect the darkness in him until it was too late.”