IMO what you need is an overview of the philosophy of mathematics/logic (as someone pointed out)
Being a philosophy, you should not expect definite answers. Mathematics itself is a historical process, so there are always trends and fashions to be aware of.
To get a quick comprehensive map, I personally always start with online articles (before books) on encyclopaedias like Plato [0] or even Wikipedia [1].
>Why certain things can be proved and others not?
One of the most captivating events of the 20th century is how Kurt Gödel's incompleteness theorems "destroyed" Hilbert’s dream of creating pure formalized foundations of all mathematics (without paradoxes and inconsistencies). I don’t think we ever fully comprehended the implications of this.
> how did Computer science or computation emerge as a branch of mathematics
Besides Gödel’s incompleteness, there is Turing's completeness. Turing Machine is the associated model of computation. These models are mathematical abstraction of computers (see "Theory of computation")
[0] https://plato.stanford.edu/entries/philosophy-mathematics/
[1] https://en.wikipedia.org/wiki/Category:Philosophy_of_mathema...