Consumers commonly incur late payment fees in the first few months of using a new credit card. These fees subsequently decline, suggesting that consumers learn to repay on time. Using a quarter of a million new credit card openings, we investigate consumer responses to penalty fees. We find that following a late payment fee, some consumers react by switching to automatic repayments, thereby insuring themselves against future forgetting and eliminating future late payment fees. Other consumers, who appear on average to be less sophisticated, continue with manual payments, relying on remembering to repay on time in the future. But this proves to be completely ineffective, with an ongoing likelihood of future fees unchanged at 20% per month. We conclude that consumer heterogeneity in adopting additional features of financial products, such as automatic repayments, is crucial for understanding whether and how consumers learn from experience.