Cross-Entropy Loss
71.3% acceptance
MediumNeural Networks
Commonly asked at:
GoogleWhy this matters:
Tests understanding of loss functions and optimization – core to training any classification model.
Cross-entropy measures the difference between two probability distributions. For classification with true labels y and predicted probabilities y_hat, implement both multi-class and binary cross-entropy loss.
Examples
Input:
y_true = [1, 0, 0], y_pred = [0.9, 0.05, 0.05]
Output:
0.105
Explanation:
Low loss because -log(0.9) ≈ 0.105
Constraints
- Add small epsilon = 1e-15 to avoid log(0)
- Handle batch inputs of shape (N, C)
- Return mean loss over batch
Tags
model-evaluationllmsrlhf