Prepare the pastry cream. Combine milk, 1/3 cup of sugar, and lemon zest in a medium saucepan. Bring the mixture to a boil. Remove from heat. In a large bowl, combine the reminder 1/3 cup of sugar with the cornstarch. Slowly whisk half of the milk mixture into the cornstarch mixture. Whisk in egg yolks, one at a time, until incorporated. Add the salt. Pour the egg yolk mixture into the milk mixture in the saucepan.
Cook the mixture over low heat, whisking continuously, until the mixture thickens. Remove from heat and add pieces of butter whisking until all the butter is incorporated and the cream looks smooth. Strain the pastry cream and let cool.
Prepare the whipped cream. In the bowl of an electric mixer fitted with the whisk attachment, whip the cream until it forms soft peaks. Slowly add sugar and vanilla. Beat at medium speed until the cream forms stiff peaks. Do not overbeat.
Assemble the trifle. Place one layer of cubes of pound cake in the bottom of a trifle dish. Top with a layer of strawberries, blueberries, and raspberries. Add dollops of pastry cream and whipped cream on top of the berries. Repeat the process ending with a layer of cake, berries, and pastry cream.
Decorate with whipped cream and whole strawberries. Place in the refrigerator and allow the trifle to rest for 20 minutes. Serve cold.