As part of the terms for Robb Stark to cross the Twins, Little Walder and Big Walder are sent to be fostered at Winterfell. Big Walder mentions Little Walder is ahead in the line of succession, and when Little Walder says neither of them will be Lord of the Crossing Big Walder says he will. They make friends with Rickon Stark, though Bran doesn't like them. When they hit the quintains Bran thinks though Little Walder strikes the harder blow, Big Walder sat his horse better. Little Walder begins mocking Hodor, Big Walder joining in slightly, however Luwin stops them. Big Walder admits what was happening, apologizes, and has the grace to look abashed, his cousin only looking peevish.
When Meera and Jojen Reed arrive Little Walder mutters insults at the Crannogmen to his cousin, the Freys land borders the Neck and the Freys have a feud with the Crannogmen.
When news arrives of the death of their half-uncle Ser Stevron Frey, heir to the Twins, the Walders show no sadness at his passing, Big Walder saying he was old and always saying he'll be tired, while Little Walder laughs. Thet are more interested in how this effects the line of succession for the Twins, Big Walder saying the new heir is Stevron's eldest son Ryman Frey, though claims he'll be lord. When Maester Luwin tells them of for being uncaring Little Walder says they're very sad unconvincingly. Though Bran doesn't like the Walders, he sends Big Walder buttered turnips at a feast and Little Walder boiled beets.
Later Theon Greyjoy takes Winterfell and one of the Walders volunteers for Theon's search party when Bran and Rickon disappear (having hidden in the Crypts though used their direwolves to leave a false trail). When Ramsay Snow returns with the Dreadfort garrison and betrays first Rodrik Cassel before they can take Winterfell, then Theon, he sacks Winterfell, and orders the Walders be spared.
Big Walder writes a letter from the Dreadfort which is signed by both Frey cousins and sent to the Twins claiming Theon Greyjoy burned Winterfell. Big Walder's uncle Lothar Frey conveys this false report to Robb Stark. Shortly after this the Red Wedding occurs, where with Roose Bolton the Freys murder Robb, his mother Catelyn Stark, and most of his army. Roose Bolton marries one of Little Walder's sisters, "Fat" Walda Frey, due to Walder offering as a dowry the weight in silver of the Frey Roose marries. Little Walder's eldest sister Amarei Frey, known as "Gatehouse Ami" for her promiscuity, is married to Lancel Lannister who has been created Lord of Darry by the Iron Throne to legitimize his hold on the Darry lands.
Big and Little Walder become squires for Roose's bastard Ramsay, who has been legitimized to Ramsay Bolton by King Tommen Baratheon to reward Roose for his treachery. Little Walder is engaged to Wylla Manderly, Wyman Manderly's younger granddaughter, the older granddaughter Wynafryd has been engaged to Rhaegar Frey, another grandson of Walder Frey. However Wyman is plotting revenge against the Freys, having lost his son Ser Wendel Manderly at the Red Wedding, though his other son Ser Wylis is a captive of the Lannisters. After Wyman's son is returned Wyman departs for Ramsay Bolton's wedding to an Arya Stark pretender, which is to be held at Barrowton. A Frey force under Big Walder's half-uncles Ser Aenys Frey and Ser Hosteen Frey has come north with Roose.
Lady Barbrey Dustin gives the Walders colts when they help Ramsay in the search for the three missing Freys who were coming to the wedding with Wyman, it is implied they were murdered which Big Walder believes happened. He tells Theon that Ramsay killed a man for calling him snow. Theon thinks Little Walder is becoming more like Ramsay, taking part in their cruel games, which worries Theon and Big Walder.
The Walders go with the Boltons to Winterfell for the wedding and are present at the ceremony. Later they join other squires in building snowmen in the yard. However a series of mysterious deaths plagues Winterfell, not helped by tensions already present between the Freys and the Northmen due to the Red Wedding. Little Walder's body is later found in a snowdrift beneath the ruined keep of Winterfell. A blood-spattered Big Walder accompanies their uncle Hosteen when he carries Little Walder's body into the Great Hall. Big Walder states Little Walder was searching for a White Harbor man who owed him silver from a dice game. Hosteen accuses Wyman of performing or ordering the murder. Wyman denies this but says "Though mayhaps this was a blessing. Had he lived he would have grown up to be a Frey." Hosteen slashes Wyman's throat, leading to a fight between the White Harbor and Frey men, leaving six White Harbor men and two Freys dead. Roose Bolton then sends both forces out against Stannis Baratheon, having received his location in a map sent by Arnolf Karstark. Theon suspects Mance Rayder and his wildling spearwives of the murder, but they claim it is no work of theirs, though they were likely behind the other murders.
- It is widely believed Big Walder murdered his cousin Little Walder. Big Walder mentions his intention to be Lord of the Crossing even though Little Walder is ahead of him in the line of succession. The fact he was blood-spattered despite the cold meaning Little Walder's blood would have frozen quickly adds to this. Big Walder may also have acted out of horror at his cousin's cruelties, the book continually contrasting the two Walders and showing Big Walder is better.