The world may be jam-packed with bedtime-themed books for young children, but I would be surprised if there were any that captured the mystery of dusk and the onset of the night like The Night Box by Louise Greig and Ashling Lindsay (Egmont, £6.99). This is the second children's book by the Scottish poet, and, like her first, The Island and the Bear, it weaves magic with every word. Max, a small boy, looks after the night, which he keeps in a box, waiting for the right moment to let it out, and put the day back in again. In this story, day yawns. It "inches like a snail around the clock". Night dances. Greig is creating books that make me wish I still had a pre-schooler to lull to sleep with her stories, and to initiate into the world of poetry with her words.

Also wonderful is Emily Gravett's Old Hat (Two Hoots, £11.99), a laugh-out-loud exhortation, by the Kate Greenaway award winner, to be yourself and not bother what everyone else is doing or wearing. In a book whose text is as funny as its absurd hat illustrations – from traffic cones to fruit bowl showstoppers – we follow Harbet, who was happy with just a cosy knitted hat till everyone else started teasing him and pushes him to try the latest fashions. It's a tale of our times, not just about clothing fashions, but also other fads – even diets. The fruit hat, for instance, boasts that it is "low in fat, high in fibre, and could provide 80% of his daily vitamins".

There's also plenty of visual humour to be found in author Becky Davies and illustrator Caroline Attia's The Three Little Pugs and the Big Bad Cat (Little Tiger Press, £6.99), an updating of the classic pigs and wolf fairy tale. The pugs, of course, are her pigs, and a demon white cat, with "scratchy claws, a terrible twitching tail and mean moggy eyes" stands in for the wolf. But, really, this book is mainly an excuse for dog-lovers to look at pugs, to croon over their cute faces and big eyes. Daft, kitsch, and wickedly funny.

Sometimes, however, it's simplicity that wins out. That's the case in one of my favourite picture books of the season, What I Think About When I Think About Running (Troika, £6.99), written by Eleanor Levenson and illustrated by Katie O'Hagan. A follow-up to the fishy tale What I Think About When I Think About Swimming, and the second of their books inspired by Haruki Murakami's What I Talk About When I Talk About Running – it muses on what might be going through a dog's mind when she/he goes out for a run (ball-play, scratching, tummy-tickles, other dogs). A great way to start a conversation about what's going on in other creatures and people's heads at an age when children are developing theory of mind.

Now is the time, however, when parents and kids are looking for some rollicking and funny holiday reads, and there are plenty out there – mostly involving wannabe superheroes. Among the funniest has to be My Evil Twin Is a Supervillain (Nosy Crow, £6.99), David Solomon's follow-up to My Brother Is a Superhero, which last year won the Children's Book of the Year at the British Book Industry Awards. Again, we follow Luke, who last time missed out on getting superpowers being doled out by an alien, because he went to the loo, and his brother got them instead. This time he is suffering the plight of having an evil twin, with super-powers, turn up from another dimension.

Meanwhile, Rachel Renee Russell, author of the wildly popular Dork Diaries, is picking up steam with the second of her new series of novels centred round a teen boy, who – guess what? – dreams of being a superhero. The Misadventures of Max Crumbly 2: Middle School Mayhem (Simon and Schuster, £10.99) begins where the last episode left off, in cliffhanger-style – on top of a Mighty Meat Monster pizza, and just goes on getting zanier from there.

This year also brings a couple of stand-out novels set in Scotland. Particularly gripping is Geraldine McCaughrean's Where the World Ends (Usborne, £9.99), a battle-for-survival tale and Scottish Lord of the Flies, inspired by the true story of a small group of men and boys who, in 1727, were marooned on remote and treacherous Stac an Armin in St Kilda. McCaughrean conjures up the wonder and hell of life on the rocky outcrop, a seabird colony, blizzarding with feathers and wings, coated in egg shell and guano, shrieking with the call of gannets.

By contrast, Elizabeth Wein's The Pearl Thief (Bloomsbury, £7.99) delivers a very different Scotland of the past, in this prequel to her hit spy-novel Code Name Verity. These days origin tales are all the rage – every spy or superhero has to have one – and The Pearl Thief delivers one, Agatha-Christie-style, as we are transported back to Julia Beaufort-Stuart's ancestral home in Perthshire, and a mystery involving mussel pearls, Traveller community, prejudice and shotguns.

But the ultimate must-reads of summer 2017 are two books which show that literature for young people is celebrating diversity, particularly in sexuality and gender, in a way that I doubt would have happened a decade ago. Indeed, you couldn't really get more diverse than Emma Donoghue's The Lotterys Plus One (Pan Macmillan, £10.99), a book that begins with the line, "Once upon a time, a man from Delhi and a man from Yukon fell in love and so did a woman from Jamaica and a Mohawk woman" and then goes on to introduce the rest of the family including seven children. Among them is Brian, a trans four-year-old, who used to be called Briar, Aspen, a 10-year-old with attention deficit disorder, and also Grumps, a very Conservative-thinking long, lost grandfather, who is moving in, has dementia, and is about to upset their progressive, home-school idyll.

It's a world away from Donoghue's chilling and disturbing bestseller Room. With its eccentric array of characters, and upbeat tone, The Lotterys Plus One may frequently feel almost the opposite of Donoghue's tale of a mother and son in captivity, but the two do share a sense that at the heart of family is language and culture. Donoghue has said that she was inspired to write it around the time her own mother was diagnosed with dementia, because she felt that most of the books out there tackling the subject for children were too dreary.

Finally, saving the best till last, Young Adult fiction's virtuoso storyteller, Patrick Ness, has published possibly his greatest and most personal novel so far in Release (Walker, £12.99). It is the coming-of-age story of Adam Thorn, 17 years old, gay, under the "Yoke" of his preacher father, and equally disapproving mother, who appear to struggle to love him for what he is, living in Washington State, and heading towards adulthood and "release". From its very first pages, Release floors with a gut-wrenching, yet funny, tender and warm, exploration of the struggle of a young man, and what it means to have to live secretly, and with disapproval and shame, and the ever-present feeling that what you are is not right.

Ness has cited his inspirations as Judy Blume's once-banned novel of teen sex and love, Forever, and Virginia Woolf's Mrs Dalloway. Release takes place, like Woolf's novel, over the course of a single day, preparing for a get together, following Thorn's memories and thoughts as he goes about his day, going for a run, being sexually harassed at work, visiting his boyfriend, and dealing with his parents – it even begins, like Mrs Dalloway, with Adam going to buy flowers. It's a daring act, taking on the format of a great classic, but Ness pulls it off, and in dazzling, heartrending fashion.