In the previous post, I have do some modification to enable the Quicknet for learning arbitrary weight sections.
However, there is still a bug which I just found it.
In the original Quicknet design, the partial weights that can be learned is only the last part. By last part, suppose for a 4 layer MLP, the parameter "mlp_lrmultiplier" is set to be "x,y,z", x control the weight between layer 1 and layer 2, y controls the weight between layer 2 and layer 3, and z controls the weight between layer 3 and layer 4.
If we set y to be 0, then no matter what value z is set, the actual z used in Quicknet is 1. That means after the first un-updated weight section, the following weights are all un-updated.
Also in the original design, if the weight is no updated, the error will not be back propagated through that layer.
All in all, in the last modification we should not remove the "break" sentences.
The new version can be found here.
However, there is still a bug which I just found it.
In the original Quicknet design, the partial weights that can be learned is only the last part. By last part, suppose for a 4 layer MLP, the parameter "mlp_lrmultiplier" is set to be "x,y,z", x control the weight between layer 1 and layer 2, y controls the weight between layer 2 and layer 3, and z controls the weight between layer 3 and layer 4.
If we set y to be 0, then no matter what value z is set, the actual z used in Quicknet is 1. That means after the first un-updated weight section, the following weights are all un-updated.
Also in the original design, if the weight is no updated, the error will not be back propagated through that layer.
All in all, in the last modification we should not remove the "break" sentences.
The new version can be found here.
No comments:
Post a Comment