The choke linkage on the throttle side of the carburetor serves two purposes: it operates the fast idle cam to engage the fast idle stop based on choke position, and serves as the "unloader" which opens the choke if the throttle is pushed all the way to the floor (to clear a flooded engine).
Because of the weight of the fast idle cam, the linkage drops as the choke opens provided the throttle is open. In other words, opening the choke lets this linkage fall DOWN. Likewise, pulling this linkage DOWN opens the choke. If the linkage is pulled DOWN and subsequently released, the tension of the choke cap should pull the cam back up so long as the throttle is open. The linkage cannot stay down unless physically held there, or pinned by the fast idle speed screw. At that point, the choke would not be able to be closed. This is why all carburetors need the pedal pushed once before a cold start - it's to unpin the fast idle cam so that the linkage is free to raise up and allow the choke to close. This all assumes that the choke tang engages the linkage under the cap and the tension is set appropriately.
It might help for you to post pictures of the linkage in question, showing the two states you describe.