Rotational Symmetry Breaking in a Trigonal sSuperconductor Nb-doped Bi2Se3

Tomoya Asaba1, B.J. Lawson1, Colin Tinsman1, Lu Chen1, Paul

Corbae1, Gang Li1, Y. Qiu2, Y.S. Hor2, Liang Fu3, and Lu Li11Department of Physics, University of Michigan, Ann Arbor, MI 48109 USA

2Department of Physics, Missouri University of Science and Technology, Rolla, MO 65409 USA3Department of Physics, Massachusetts Institute of Technology, Cambridge, MA 02309 USA

(Dated: December 25, 2016)

The search for unconventional superconductivity has been focused on materials with strong spin-orbit coupling and unique crystal lattices. Doped bismuth selenide (Bi2Se3) is a strong candidategiven the topological insulator nature of the parent compound and its triangular lattice. Thecoupling between the physical properties in the superconducting state and its underlying crystalsymmetry is a crucial test for unconventional superconductivity. In this paper, we report directevidence that the superconducting magnetic response couples strongly to the underlying trigonalcrystal symmetry in the recently discovered superconductor with trigonal crystal structure, niobium(Nb)-doped bismuth selenide (Bi2Se3). As a result, the in-plane magnetic torque signal vanishesat every 60◦. More importantly, we observed that the superconducting hysteresis loop amplitudeis enhanced along one preferred direction spontaneously breaking the rotational symmetry. Thisobservation confirms the breaking of the rotational symmetry and indicates the presence of nematicorder in the superconducting ground state of Nb-doped Bi2Se3.

Unconventional superconductors are characterized bysuperconducting order parameters that are non-invariantunder crystal symmetry operations. When the order pa-rameter is single-component, this non-invariance is man-ifested solely in the phase of the superconducting wave-function, and can only be detected by phase-sensitivemeasurements. On the other hand, when the order pa-rameter is multi-component, the magnitude of the super-conducting gap can be different along symmetry-relatedcrystallographic directions. The gap anisotropy leads di-rectly leads to a thermodynamic property of the super-conducting state that spontaneously breaks the crystalrotational symmetry of the normal state. Prior to thisworkHowever, no direct thermodynamic signature of ro-tational symmetry breaking due to superconductivity hasnot been observedfound in any crystals.

Bismuth selenide (Bi2Se3) makesis one of the bestmostpromising materials system to explore for unconventionalsuperconductivity. The sStrong spin-orbit coupling inthe triangular lattice has led to theproduces a topolog-ically insulating ground state in Bi2Se3 [1, 2]. Dopingwith a metallic element such as copper (Cu), and stron-tium (Sr), or niobium (Nb) mademakes it superconduct-ing [3–11]. We report here the first direct observa-tion of rotational symmetry breaking in the supercon-ducting propertystate of Nb-doped Bi2Se3 [12], a newmember of the superconducting doped topological insu-lators in addition to Cu- and Sr-doped Bi2Se3. Possi-ble odd-parity pairing symmetries in doped Bi2Se3, fa-vored by strong spin-orbit interactions, have been theo-

Corresponding authors: L. Li (luli@umich.edu), Y. Hor(yhor@mst.edu)

retically proposed and classified according to the repre-sentations of the D3d point group [5]. Among them, onlythe odd-parity pairing in the two-dimensional Eu rep-resentation gives rise to a nematic superconductor withbroken rotational symmetry [13]. A recent nuclear mag-netic resonance experiment on CuxBi2Se3 reveals an in-plane anisotropy in the spin susceptibility of the super-conducting state [14], providing spectroscopic evidencefor the Eu pairing. An electrical transport study onSr-doped Bi2Se3 reveals the anisotropy of upper criti-cal field Hc2 [16]. Heat capacity measurements also showthe anisotropy of Hc2 [15]. Yet, there is not clear ev-idence showing the anisotropy in the superconductingproperties deep in the superconducting state at H muchsmaller than Hc2. To solve this problem, we chose anunique method of applying, torque magnetometry, to re-solve the anisotropy of the superconducting critical cur-rent in these materials.

Our torque magnetometry approach is quite a newand powerful tool to investigate the nematicity. In gen-eral, rotational symmetry breaking measurements requirenon-contact methods such as heat capacity, magnetiza-tion or NMR since contact itself may generate a preferredaxis. However, the combination of field rotation and ther-mal measurements or NMR requires special effort, whiletorque magnetometry can be performed in a rather stan-dard and direct setup. In addition, we focus on the sym-metry of the hysteresis loop as well as magnetic suscep-tibility in the superconducting state. As far as we know,this is the first time to show that torque magnetometrycan behas been used to investigate the nematic order inthe superconducting state. By contrast, ittorque mag-netometry has been used for detectingto detect nematic-ity only in the normal state in superconductors URu2Si2

2

or BaFe2(As1−xPx)2 [18, 19]. Thus, our new approachpaves a newthe way for the further studies on nematicityin the superconducting ground state.

More importantly, observing the nematicity in thesuperconductivitying state itself is an exciting result. Asmentioned above, observing nematicity in URu2Si2 orandBaFe2(As1−xPx)2 havehas had a huge impact on the fieldsince it significantly helps theus to understanding of theunconventional and complicated behaviors of those su-perconductors. Similarly, rotational symmetry break-ing in the superconducting state suggests spin tripletpairing and topological superconductivity. Thus, ourapproach and results would greatly help understandingthegreatly impact future research on topological super-conductivityor research.

The detailed growth method of Nb-doped Bi2Se3 singlecrystals is described in [12]. Crystal orientation is deter-mined by the X-ray diffraction on single crystals. TheX-ray diffraction pattern is known to be almost identicalto Bi2Se3 [17]. The crystal orientation was determinedfrom aA 2θ = 43.7◦ peak corresponding to the (110) planewas observed from certain crystal plane to determine thecrystal orientation.

We used torque magnetometry to measure the su-perconducting hysteresis loop and magnetization of Nb-doped Bi2Se3. Since the free energy with respect to theexternal field H is given by F = µ0V ~M · ~H, magnetictorque is given by the derivative of F with respect to themagnetic field tilt angle φ: ~τ=µ0V ~M × ~H. Here V is thevolume of the sample, ~H is the external magnetic field,and ~M is the magnetization of the sample. Torque mag-netometry is thus a thermodynamic probe that measuresthe free energy directly.

The torque is measured by mounting the sample stand-ing on its edge in order to keep the external field inthe sample’s ab-plane (see Fig. 1(A)). We then rotatedthe cantilever. This measures the in-plane anisotropy ofthe sample’s magnetic properties in the superconductingstate. In general case, the dominating term of paramag-netic torque is a periodic function of double the azimuthalangle φ:

τ2φ =1

2µ0VMH2{(χaa−χbb) sin 2φ− 2χab cos 2φ}. (1)

Here Mi = ΣjχijHj . In a system with high symmetry(e.g., tetragonal or hexagonal), τ2φ becomes 0 becauseχaa = χbb and χab = 0. In such a case, the leadingtorque term becomes

τ2nφ = A2nφ sin 2n(φ− φ0), (2)

where n is determined by the crystalline symmetry: e.g.,n = 2 for the tetragonal lattice or n = 3 for the hexagonallattice. For tetragonal URu2Si2 or BaFe2(As1−xPx)2 [18,19], sin 4φ is dominant at high temperature while sin 2φbecomes pronounced at low temperature, indicating the

emergence of nematic order at low temperature. In ourcase, Fig. 1(B) shows the crystal structure of Nb-dopedBi2Se3 looking down the hexagonal axis. As shown inthe figure, the external field is in the hexagonal plane.Thus, in a paramagnetic normal state, a sin 6φ torqueresponse is expected from the crystal structure. In thiscase the anisotropy ofin the magnetic susceptibility wouldbring the magnetic torque to zero along the in-plane mir-ror lines and the normal axis of the mirror lines of thecrystals. The azimuthal angle, φ, is the angle between theexternal magnetic field and the x-axis defined alongby thecantilever arm. Based on the X-ray diffraction patternof this particular sample, we find that φ = 0◦, 60◦, and120◦ corresponds to the in-plane mirror line of the crys-tal, as shown asby the dashed lines in Fig. 1(B). In thesame crystal structure diagram, the blue axes are the in-plane crystal axes, which are 30 degrees away from themirror lines. Furthermore, a dominant sin 2φ indicatesrotational symmetry breaking.

The samples of Nb-doped Bi2Se3 sample used in ourexperiment have has a superconducting volume close to100%, as shown by the volume magnetic susceptibilitywhich approaches -1 in the zero-field cooled run (see Fig.1(C)). This is much higher than that of Cu-doped Bi2Se3[8, 9].

The measured torque shows a strong superconductingsignal. Figure 1(D) shows some examples of the magnetictorque on the cantilever from the sample. The torque τis plotted as a function of the external magnetic field attemperate T = 0.3 K. We swept the field up and downfrom -1 T to 1 T to measure the entire superconductinghysteresis loop. Arrows along the curve show the fieldsweep direction. The τ − H loop is a signature of thestrong flux pinning characteristic of type-II superconduc-tors. The pinned flux lines form a vortex solid, and theflux density inside the superconductor always resists thechange ofin the applied magnetic field. A simple analysisbased on the Bean model shows that the hysteresis of themagnetization gives a direct measurement of the super-conducting critical current density in the mixed state oftype-II superconductors (see supplemental materials).

To probe the symmetry of the superconducting sam-ple, we focused on both the symmetry of magnetic sus-ceptibility and the hysteresis loop. We simply defineM± = τ±/µ0H, M = (M+ + M−)/2 and ∆Meff =(M+ −M−)/2, as well as the effective magnetic suscep-tibility χ = dM

µ0dHwhere τ+ (τ−) are the sample’s mag-

netic torque from the H-increasing (decreasing) sweeps,respectively (Fig. 2(A)). Since the critical field Hc2 isabout 0.6 T, above 0.6 T the sample enters the normalstate and τ+ and τ− (so asalso M+ and M−) overlap witheach other. Below the critical field, M = (M+ +M−)/2is the average magnetization from the mixed state. Fromthe Bean model, supercurrents induced by field sweep-ing up and field sweeping down contribute equally, soM = (M+ + M−)/2 corresponds to the intrinsic mag-

3

netic susceptibility fromof the sample. The sample wasmeasured at T = 0.3 K over an angular range of 200◦.Since the geometry demands that rReversing the sign ofthe magnetic field is equivalent to rotating the cantilever180◦, so we extend the negative field sweep data to com-plete the 360◦ angular dependence.

We first check the crystalline symmetry in the nor-mal state at high fields. In the normal state, the ef-fective magnetic susceptibility χ shows a periodic pat-tern with a period of 60◦. Note that, while the actualanisotropic susceptibility is constant, the effective sus-ceptibility χ = dM

µ0dHis multiplied by the angular factor

sin 2nφ, resulting in zero or negative χ. As stated abovein Eq. 2, this 60◦ period is expected from the hexagonallattice of the samples. Fig. 2(B) shows the angular de-pendence of the susceptibility χ at 0.8 T, well above thecritical field. As shown clearly, tThe normal state torqueclearly follows thea periodic function of 6φ. The FastFourier Transformation (FFT) of the data ofin Fig. 2(B)is shown in Fig. 2(C). As shown, the Fourier componentof 6φ is dominant in the normal state. If the samplesymmetry is broken, the contribution from the 2φ com-ponent should be orders of magnitude larger. Therefore,a small 2φ component would be from the higher orderterm of the geometric factors. Thus, we conclude thatthe sample is hexagonal in the normal state.

Now, we focus on the symmetry of the superconduct-ing state. In a superconducting state, the torque curvecannot be simply explained by the formula for thea para-magnetic systems. However, from analogue thatby anal-ogy to the torque response of a hexagonal ferromagneticmaterial, is still τ = A6φ sin 6φ, the torque curve fromthe hexagonal superconducting material should followthe sin 6φ dependence as well.

By contrast, the angular dependence of our dataclearly breaks the sin 6φ pattern. Fig. 2(D) shows theangular dependence of the magnetic susceptibility at 0.05T, well below the critical field. It is clearly shown thatthe angular dependence of the susceptibility is differentfrom the normal state. Fig. 2(E) shows the FFT of thedata ofin Fig. 2(D). Most importantly, Notice that the 2φand 4φ components becomeare dominant instead ofratherthan the vanished 6φ component. This implies that thesuperconducting state breaks the rotational symmetry re-quired by the crystalline lattice.

Finally, the symmetry breaking in the superconduct-ing state is even more pronounced in the hysteresisloop. Figure 3(A) shows the magnitude of the hysteresismagnitude of the magnetization ∆Meff = (M+−M−)/2.The hysteresis loop size, of ∆τ∆Meff , goes to zero ev-ery 60◦. We also would like to note that for a broadfield range the absolute amplitude of A2φ and A4φ areequal as shown in the FFT data (Fig. 3(B) and (C)).AtUnder this condition, the ∆Meff follows the func-tion of f(φ) = 2A2φsin(φ− 30◦) cos 3φ, as shown as abythe solid line in Fig. 3(A). This fitting reveals a possi-

ble origin of the observed rotational symmetry break-ing. The effective superconducting magnetic momentfollows the product of sin(φ − 30◦) and cos 3φ, ratherthan the sum of two ordering functions. Therefore, thereis a strong coupling between the crystalline symmetryand a nematic ordering in the triangular superconductorNb-doped Bi2Se3. The phases between these two sinu-soidal functions are locked in the measurement. Thissuggests that while nematic order is a spontaneous sym-metry breaking in the superconducting state, the orderseems to be stabilized by the crystalline symmetry. Thecoupling makes sure that the nematic ordering directionis locked to one of the mirror planes of the triangular lat-tice. As pointed out in ref. [13], the nematic order verifiesthe two-component nature of the superconducting orderparameter. Thus an odd-parity superconducting orderis very likely to exist in the ground state, which createspromise for topological superconductivity in Nb-dopedBi2Se3.

Discussion We discuss first the advantages and disad-vantages of our approach. As mentioned above, torquemagnetometry does not require special instruments ex-cept an in situ rotator. This method has been usedfor observing the normal state symmetry breaking ofstrongly correlated materials [18, 19] assince it is verysensitive to the crystal symmetry breaking. In ourstudy, we show that the amplitude of the hysteresis looporiginating from the supercurrent reflects the symme-try of the magnetic susceptibility in the superconductingstate. Generally the hysteresis signal is much larger thanthe signal from magnetic torque in the superconductingstate, so focusing on the symmetry of the hysteresis loopsymmetry will beis a good way ofto detecting the sym-metry breaking in small samples.

There are disadvantages as well ofin using torquemagnetometry resolvingto detect symmetry breaking.Firstly, torque magnetometry isdoes not a directly toolto measure the superconducting gap symmetry. Thus,even though nematic order is observed by torque, furtherexperiments such as STM are required to exactly under-stand exactly the origin of the nematicity. Secondly, sincethe anisotropy between in-plane and out-of plane is oneorder of magnitude larger than within the plane, a smallangle misalignment of the sample could affect the mea-surement. As shown in the supplement, when the sampleis misaligned by a few degrees, it shows extra contribu-tion of the A2φ component. Thus, extra attention has tobe paid to exclude the possibility that A2φ is purely fromthe sample misalignment. However, note that even withthe additional contribution, the measurement still revealstwo key nematic features: (a) a large A4φ component and(b) vanishing A2φ component with high fields.

Next, our study of the magnetic torque demonstratesa symmetric vanishing of the superconducting hysteresis.Magnetic torque is sensitive to the magnetic anisotropyof the superconducting signal, and the hysteretic M −H

4

loops arise from the flux pinning of the vortex solidsin the superconducting state. Therefore, the supercon-ducting diamagnetic signal, which comes from the vortexsolid, prefers to align along the crystalline axis planes ofthe triangular lattice. As shown by the crystal structureplot in Fig. 1, the crystalline axis planes exist at every60 degrees. The diamagnetic signal is maximum whenthe magnetic field aligns with these preferred directions.The magnetic torque ~τ = µ0

~M× ~H vanishes at these pre-ferred directions when the M vector is exactly collinear(parallel or antiparallel) of the applied field H.

A question arises as to why the diamagnetic responseis preferred on these particular axes. As pointed outin ref. [13], the multi-component superconducting orderparameter leads to a nematic superconducting state thatspontaneously breaks rotational symmetry of the crystaland exhibits uniaxial anisotropy in the ab−plane. As aresult, it is expected that the superconducting diamag-netic signals sticks to these preferred axes.

The hysteresis loop is greatly enhanced along one di-rection. To explore possible explanations for the rota-tional symmetry breaking, we first consider extrinsic fac-tors that can induce symmetry breaking. Firstly, if thiswas an electrical or heat transport measurement, it couldbe from the measurement current or heat flow itself.However, torque magnetometry is a non-contact mea-surement, and does not break the symmetry of the bulksample. Indeed, this is one of the big advantages of ourmethod, as torque measurement can be easily combinedwith field rotation. Secondly, structural phase transitionis not likely the origin of the observed rotation symmetrybreaking. We observed the smooth heat capacity and re-sistivity as a function of temperature (see supplementalmaterials). The symmetry of the torque measurementin the normal state also shows sin6φ dependence as ex-pected by the crystalline symmetry, which indicates thatthe rotational symmetry breaking originates from the su-perconducting state.

It is also possible that the spontaneous enhanced di-rection is locked to the strain or structure defects. Themagnetization hysteresis loop measures the supercurrent.Thus the hysteresis loop is determined by the domainstructures. The domain pattern might be pinned to somedirections by the crystalline defects. We note that, how-ever, the effective magnetic susceptibility χ also showsthe rotational symmetry breaking behavior in the super-conducting state. Given that χ averages from the upand down sweeps, based on the Bean model, χ shouldnot couple to the domain formation. Thus the rota-tional symmetry breaking in χ suggests that the do-main structures do not play a key role in rotationalsymmetry breaking. Furthermore, a periodic vanish-ing of the hysteresis loop requires unconventional order-ing in the superconducting state. Early torque studieson high Tc cuprates, untwinned YBa2Cu3O7 [20] andTl2Ba2CuO6+δ [21], show in-plane vanishing of the su-

perconducting hysteresis along their crystal axes. Incontrast, in classical s-wave superconductors a similarmeasurement on NbN and NbSe2, which are classicalsuperconductors with s-wave symmetry [21], the torquesignal with H in plane completely vanishes in all thedirectionsshows no anisotropy [21].

Therefore we conclude that the symmetry breaking isan intrinsic behavior. There are several possible origins.The first one is odd parity superconductivity with a node-less superconducting gap. As shown in the supplement,our heat capacity measurement is consistent with theanodeless gap. However, since the superconducting vol-ume is not 100 %, there is a remaining density of stateswhich may affect the shape of heat capacity curve. Thesecond possibility is a nodal even-parity superconductinggap with Eg order [13]. Recently it has been reportedthat the penetration depth shows a power-law behaviorat low temperatures, indicating point nodes [22]. Furtherexperiments such as STM measurement are required todetermine whether there is a nodal gap structure of theinthis material.

We would like to also point out a potential differencebetween our observations in Nb-doped Bi2Se3 and thesimilar early experimental observation onin Cu-dopedBi2Se3[14, 15], Sr-doped Bi2Se3[16] and the theoreticalexplanation [13] of nematic order in the spin susceptibil-ity of Cu-doped Bi2Se3. In the Cu-doped material, quan-tum oscillations [7, 9] and photoemission [6, 11] revealthat there is only one bulk Fermi pocket. In contrast, theelectronic state in Nb-doped Bi2Se3 shows at least twodistinct Fermi surfaces [23]. Our work calls for furthertheoretical and experiential exploration of the impact ofmulti-orbitals on the search for unconventional supercon-ductors. While in the Sr-doped materialBi2Se3 large Hc2

anisotropy has been observed between field parallel andperpendicular to the crystal axis [16], our torque showsrelatively isotropic critical field. This difference couldbe due to the measurement technology as torque magne-tometry is a thermodynamic probe, but f. Further studywould be required to elucidate this question.

The unique sensitivity of our in-plane torque magne-tometry leads to a promising new tool for probing the su-perconducting pairing symmetry of other unconventionalsuperconductors. It can elucidate or confirm potentialp-wave superconducting symmetry in materials such asSr2RuO4 [24] and UPt3 [25, 26]. In contrast to thosematerials, the exponential decay in the heat capacity inNb-doped Bi2Se3 suggests a superconducting gap with-out line nodes. It would be very interesting to investigatewhether a nodal superconducting gap would enhance ordiminish the nematic order in the superconducting state.

5

MATERIALS AND METHODS

We preformed torque magnetometry measurementswith our home built cantilever setup by glueing the Nb-doped Bi2Se3 single crystal to the end of a thin berylliumcopper cantilever. We then placed the cantilever in anexternal magnetic field H. We measured the torque onthe cantilever by tracking the capacitance between themetallic cantilever and a fixed gold film underneath us-ing an AH2700A capacitance bridge with a 7 kHz drivefrequency. We calibrated the spring constant of the can-tilever by tracking the angular dependence of capacitancecaused by the sample weight at zero magnetic field.

The tilt angle φ is defined as the angle between thedirection of the magnetic field and the positive x-axis,which is marked in Fig. 1(A) as the direction of thecantilever arm.

The sample heat capacity is measured in a Quan-tum Design Physical Properties Measurement Systems(PPMS) using the relaxation method.

The National High Magnetic Field Laboratory pro-vided the magnet and He3 fridge. During the torquemagnetometry measurement, samples were soaked inpumped liquid helium 3, and the magnetic field was sweptat 0.25 T/min.

We preformed the magnetization measurement with aQuantum Design Magnetic Properties Measurement Sys-tem at H = 5 Oe.

Acknowledgements This work is mainly supported bythe Department of Energy under Award No. DE-SC0008110 (magnetization measurement), by the Na-tional Science Foundation under Award No. DMR-1255607 (sample growth), the David and Lucile Packardfoundation (theory). Supporting measurements weremade possible with the support by the Office of NavalResearch through the Young Investigator Prize underAward No. N00014-15-1-2382 (thermodynamic charac-terization), by the National Science Foundation underAward No. 1307744 (electrical transport characteriza-tion), and the National Science Foundation Major Re-search Instrumentation award under No. DMR-1428226(supports the equipment of the thermodynamic and elec-trical transport characterizations). Some experimentswere performed at the National High Magnetic Field Lab-oratory, which is supported by NSF Cooperative Agree-ment No. DMR-084173, by the State of Florida, and bythe DOE. We are grateful for the assistance of Tim Mur-phy, Glover Jones, and Ju-Hyun Park of NHMFL. T.A.thanks the Nakajima Foundation for support. B.J.L.acknowledges support by the National Science Foun-dation Graduate Research Fellowship under Grant No.F031543.

[1] M. Z. Hasan and C. L. Kane, Rev. Mod. Physics. 82, 3045(2010).

[2] S.C. Zhang and X. L. Qi, Rev. Mod. Phys. 83, 1057 (2011).[3] Y. S. Hor, A. J. Williams, J. G. Checkelsky, P. Roushan,

J. Seo, Q. Xu, H. W. Zandbergen, A. Yazdani, N. P. Ong,and R. J. Cava, Phys. Rev. Lett. 104, 057001 (2010).

[4] Zhongheng Liu, Xiong Yao, Jifeng Shao, Ming Zuo, Li Pi,Shun Tan, Changjin Zhang, and Yuheng Zhang, Journalof the American Chemical Society 137, 33 (2015).

[5] L. Fu and E. Berg, Phys. Rev. Lett. 105, 097001 (2010).[6] L. A. Wray, S.-Y. Xu, Y. Xia, Y. S. Hor, D. Qian, A. V.

Fedorov, H. Lin, A. Bansil, R. J. Cava, and M. Z. Hasan,Nat. Phys. 6, 855 (2010).

[7] B.J. Lawson, G. Li, F. Yu, T. Asaba, C. Tinsman, T. Gao,W. Wang, Y.S. Hor, and Lu Li, Phys. Rev. B 90, 195141(2014).

[8] M. Kriener, K. Segawa, Z. Ren, S. Sasaki, and Y. Ando,Phys. Rev. Lett. 106, 127004 (2011).

[9] B. J. Lawson, Y. S. Hor, and Lu Li, Phys. Rev. Lett. 109,226406 (2012).

[10] S. Sasaki, M. Kriener, K. Segawa, K. Yada, Y. Tanaka,M. Sato, and Y. Ando, Phys. Rev. Lett. 107, 217001(2011).

[11] E. Lahoud, E. Maniv, M. S. Petrushevsky, M. Naamneh,A. Ribak, S. Wiedmann, L. Petaccia, Z. Salman, K. B.Chashka, Y. Dagan, and A. Kanigel, Phys. Rev. B 88,195107 (2013).

[12] Yunsheng Qiu, Kyle Sanders, Jixia Dai, Julia Medvedeva,Weida Wu, Pouyan Ghaemi, Thomas Vojta, and Y.S. Hor,arXiv:1512.03519 (2015).

[13] L. Fu, Phys. Rev. B 90, 100509(R) (2014).[14] K. Matano, M. Kriener, K. Segawa, Y. Ando, Guo-qing

Zheng, Nat. Phys. doi:10.1038/nphys3781 (2016)[15] Shingo Yonezawa, Kengo Tajiri, Suguru Nakata, Yuki

Nagai, Zhiwei Wang, Kouji Segawa, Yoichi Ando andYoshiteru Maeno, Nat. Phys. doi:10.1038/nphys3907

[16] Y. Pan, A. M. Nikitin, G. K. Araizi, Y. K. Huang, Y.Matsushita, T. Naka and A. de Visser, Scientific Reports6, 28632 (2016)

[17] I. K. Robinson and D. T. Tweet, Rep. Prog. Phys. 55,599 (1992).

[18] S. Kasahara et al., Nature (London) 486, 382 (2012).[19] R. Okazaki et al., Science 331, 439 (2011).[20] T. Ishida, K. Okuda, H. Asaoka, Y. Kazumata, K. Noda,

and H. Takei, Phys. Rev. B 56, 11897 (1997).[21] M. Willemin, C. Rossel, J. Hofer, H. Keller, Z. F. Ren,

and J. H. Wang, Phys. Rev. B 57, 6137 (1998).[22] M. P. Smylie, H. Claus, U. Welp, W.-K. Kwok, Y. Qiu,

Y. S. Hor and A. Snezhko, arXiv:1608.08164 (2016)[23] B.J. Lawson, Paul Corbae, Gang Li, Fan Yu, Tomoya

Asaba, Colin Tinsman, Y. Qiu, Y.S. Hor, and Lu Li, underreview.

[24] A. P. Mackenzie and Y. Maeno, Rev. Mod. Phys. 75, 657(2003).

[25] J. A. Sauls, Adv. Phys. 43, 113 (1994).[26] J.D. Strand, D.J. Bahr, D.J. Van Harlingen, J.P. Davis,

W.J. Gannon, and W.P. Halperin, Science 328, 1368(2010).

6

(A)

(B)

(C)

(D)

y

x

HSe

Bi

x

y

HM

Torque τ

φ

φ

FIG. 1. Experimental setup, sample orientation andtorque curves of NbxBi2Se3. (color online) (A) Schematicsketch of torque magnetometry under in-plane field rotation.The magnetic field is applied in-plane. The azimuthal an-gle φ is defined as the angle between the external magneticfield and the cantilever arm (x-axis). Torque ~τ=µ0V ~M × ~His tracked by measuring the capacitance between the can-tilever and the gold film beneath it. (B) Crystal structureof NbxBi2Se3 viewed down the crystalline c-axis. The dashedlines are the mirror planes of the crystal, and the blue arrowsare the crystal axes. These crystal axes are shown to be thedirections where the torque loop vanishes. (C) Meissner effectfrom the sample. The volume magnetic susceptibility reachesclose to -1, indicating a fully superconducting volume. (D)Selected torque curves at 0.3 K with external magnetic fieldbetween -1 and 1 T. The magnitude of the hysteresis loop ismaximum at around 120◦ and is nearly zero at 30◦ and 90◦.

7

0 1 2 3 4 5 6 7 8 9 1 002 04 06 08 0

1 0 01 2 0

FFT A

mplitu

de (a

.u.) o

f dM/

dH

H a r m o n i c s o f n

A 2 φ

T = 0 . 3 KB = 0 . 0 5 T

A 4 φ

- 6 0 0 6 0 1 2 0 1 8 0 2 4 0 3 0 0 3 6 0

- 1 0 0

- 5 0

0

5 0

1 0 0

1 5 0

χ = dM

/dH (1

0-9 Nm/T2 )

A n g l e ( d e g r e e )

0 . 0 5 T - 0 . 0 5 T

- 1 . 0 - 0 . 5 0 . 0 0 . 5 1 . 0- 4 0- 3 0- 2 0- 1 0

01 02 03 04 0

µ0 H ( T )

M + = τ+/ µ0 H M - = τ−/ µ0 H M = ( M + + M - ) / 2

M = τ

/µ 0H (1

0-9 Nm/T)

χ = d M / d H

∆M e f f = ( M + - M - ) / 2

φ= 1 2 1 o

0 1 2 3 4 5 6 7 8 9 1 0012345678

FFT A

mplitu

de (a

.u.) o

f dM/

dH

H a r m o n i c s o f n

A 6 φ

T = 0 . 3 KB = 0 . 8 T

0 6 0 1 2 0 1 8 0 2 4 0 3 0 0 3 6 0

- 5

0

5

χ = dM

/dH (1

0-9 Nm/T2 )

A n g l e ( d e g r e e )

0 . 8 T - 0 . 8 T

( E )( D )

( C )( B )N o r m a l s t a t e

M i x e d s t a t e

( A )

FIG. 2. The angular dependence of susceptibility of NbxBi2Se3 in a normal state and superconducting state.(color online) (A) Examples of the hysteretic M -H curves demonstrates the definition of M , ∆M , and χ. M± = τ±/µ0H,M = (M+ +M−)/2, ∆M = (M+−M−)/2 and χ = dM/µ0dH, where τ+ is the torque signal from the up-sweep of the magneticfield, and τ− is the torque signal from the down-sweep of the magnetic field. (B)Angular dependence of effective susceptibilityχ in a normal state at 0.3 K. Magnetic field of 0.8 T was applied to suppress the superconductivity. Angle φ is defined as thetilt angle between the positive x-axis and the magnetic field direction. Reversing the sign of the magnetic field is equivalent torotating the cantilever 180◦. We thus used the negative field values to complete the 360◦ angular dependence (open circles).It is clearly shown that sin 6φ is dominant. This indicates that the crystal has hexagonal symmetry. (C) The Fast FourierTransform (FFT) plot of the data shown in (B). A6φ is dominant while A2φ and A4φ are very small. (D) Angular dependenceof effective susceptibility χ in a superconducting state at 0.3 K. The crystal symmetry is lower than normal state, resulting ina lower n periodic function of φ. (E) FFT plot of the data shown in (D).A2φ and A4φ are dominant while A6φ is very small.The contrast between the pattern in the normal state and the superconducting state demonstrate the breaking of the rotationalsymmetry in the superconducting state.

8

0 6 0 1 2 0 1 8 0 2 4 0 3 0 0 3 6 0- 3 0- 2 0- 1 0

01 02 03 04 0

∆Meff

= (M +-M

-)/2 (1

0-9 Nm/T)

A n g l e ( d e g r e e )

0 . 0 1 T 0 . 1 0 . 2 0 . 3 0 . 4 0 . 5

0 5 1 00 . 0 0 00 . 0 0 10 . 0 0 20 . 0 0 30 . 0 0 40 . 0 0 5

0 . 0 0 . 1 0 . 2 0 . 3 0 . 4 0 . 5 0 . 6 0 . 7 0 . 81 E - 5

1 E - 4

0 . 0 0 1

A 4 φ µ0 H = 0 . 0 5 T

FFT A

mplitu

de (a

.u.)

H a r m o n i c s o f n

A 2 φ

A 2 φ

A 4 φ

( C )FF

T amp

litude

µ0 H ( T )

( B )( A )

FIG. 3. The angular dependence of spontaneous effective magnetization. (color online) (A)The angular dependenceof spontaneous effective magnetization ∆M − φ. The data were taken at 0.3 K at a few selected H fields. Data taken fromthe positive field sweep are plotted as filled symbols, and data from the negative field sweep are plotted as open symbols. Thesolid lines show f(φ) = 2A2φsin(φ − 30◦) cos 3φ. (B) Fast Fourier Transform (FFT) of ∆M − φ at µ0H = 0.05 T. The firstpeak, A2φ, is the amplitude of the nematic order term. The second peak, A4φ, represents the sin 4φ term originated from theproduct fit function f(φ) = 2A2φsin(φ− 30◦) cos 3φ. (C) The magnetic field dependence of the the FFT amplitudes A2φ andA4φ in the superconducting state of Nb-doped Bi2Se3. The FFT amplitude is plotted in logarithmic scale for clarity. Above0.6 T, the superconducting hysteresis loop quickly vanishes as H approaches the upper critical field.

S1

SUPPLEMENTAL MATERIALS

Superconducting Hysteresis

An external magnetic field kills the superconductingstate in a type I superconductor at the critical field, Hc.However, in a type II superconductor, there is a mixedstate between the lower critical field, Hc1, and the up-per critical field, Hc2. For Hc1 < H < Hc2, magneticflux penetrates the superconductor creating a lattice ofvortices. Due to defects in the sample, these vorticesare pinned in place. In order for the vortices to move,the Lorentz force from a current near the vortices wouldneed to overcome this pinning force. Thus, the pinnedmagnetic flux has an irreversible response to a changingexternal magnetic field. This gives rise to hysteresis inthe magnetic response of the superconductor.

The Bean model [1, 2] successfully explained this hys-teretic feature in type II superconductors. In this model,we assume that the current density in the superconductorcan only take the values of 0 and Jc, where Jc is the criti-cal current density. Due to the Ampere’s law, the spacialprofile of Jc determines b(x), the magnetic flux densityof unit volume at each location x in the superconductor.Integrating b(x) gives the total magnetic field density Binside the superconductor.

H=0

H0

Hincreasing

Hdecreasing

H0

H>H0

Jc

Jc

(A)

(B)

b(x)/μ0

b(x)/μ0

UpsweepJcprofileatH0

DownsweepJcprofileatH0

FIG. S1. Schematic of magnetic flux density in type IIsuperconductors (color online) (A) Magnetic flux density ina type II superconductor as external magnetic field is sweptup from H = 0 to H0 > 0 according to the Bean model. Theright inset shows a sketch of the critical current density profileat H0 during the upsweep. Magnetization M corresponds tothe dark shaded area. (B) Magnetic flux density in a type IIsuperconductor as external magnetic field is swept down fromH > Hc2 to H0. The lagging of the internal magnetic fluxdensity due to flux pinning gives rise to hysteresis in magne-tization. The sample’s critical current density profile for thedown sweep is shown in the right panel. Figure adapted fromRef. [3].

Figure S1 shows the magnetic flux density inside thesuperconductor (represented by the grey shaded region)for (A) field increasing from H = 0 to H0 and (B) fielddecreasing to H0 after an applied external field greaterthan Hc2. Due to the flux-pinning, the internal mag-netic flux density b(x)/µ0 cannot respond to the changein the sweep direction of the external magnetic field. Fig.S1(A) represents the magnetic flux density b(x)/µ0 as theapplied magnetic field is swept up from zero to H0. In

this one dimensional analysis, Jc = 1µ0∇b(r) = 1

µ0

db(x)dx

would be simply the slope of b(x)µ0vs. x. The spacial pro-

file of Jc is constant, as shown in the right panel of FigS1(A).

The situation of field-sweeping-down is different. Asthe applied field is swept from a large field to the sameH0, the b(x)/µ0 profile response lags, as show in Fig.S1(B).

The magnetization of the sample is given by the dif-ference between the average magnetic flux density withinthe sample, B, and the applied field, H, outside:

M =B

µ0−H =

1

w

∫dxb(x)

µ0−H (S1)

where w is the typical width of the domain size in thesuperconductor, or the sample size if the whole sampleis in a single domain. In the case of Panel A, Eq. S1would be simply the shaded area −wJc which gives M+,the magnetization at field sweeping up.

Furthermore, the difference between Panel A and Bdemonstrates the hysteresis in the magnetization of typeII superconductors. Going through the same analysis asabove, we find the magnetization of field sweeping downM− = wJc As a result, the hysteresis loop size ∆M(H) ≡M+ −M− = 2wJc. Therefore, the measurement of thehysteresis loop is a direct probe of the critical currentdensity in a type II superconductor.

Heat capacity of Nb-doped Bi2Se3

Our heat capacity measurement on Nb-doped Bi2Se3shows a fully gapped bulk superconductivity (Fig.S2(A)). For the same sample, we measured the heat ca-pacity C at selected T between 0.4 K and 20 K. Fig.S2(A) shows C

T in the superconducting state at µ0H =0 T as well as at 0.75 T, right above the closing of thehysteresis loop at base T . We note that above 4 K, theheat capacity C at 0 T is the same as that at 0.75 T,within measurement errors. Therefore, we use the 0.75T heat capacity curve as the normal state heat capacityCn. From this we determined the phonon contributionfollowing the same practice as the early work on Cu-doped Bi2Se3 [8]:

Cn = Cnel + Cph = γnT + aT 3 + bT 5 (S2)

S2

where the electronic heat capacity Cnel = γnT is for thenormal state. Subtracting the phonon heat capacity, Cph,we infer the superconducting state electronic heat capac-ity Cel at 0 T, which is plotted as Cel

T vs. T in Fig. S2(B).The heat capacity at 0 T shows an exponential decay asT drops to base temperature. The complete Cel

T vs. Tis consistent with the fit based on BCS theory. The the-oretical fit conserves the entropy in the superconductingtransition.

0 1 2 3 40

1 02 03 04 05 06 0

0 1 2 3 40123456789

1 01 11 2

C/T(m

J/molK

2 )

T ( K )

µ0 H = 0 T µ0 H = 0 . 7 5 T

0 T

0 . 7 5 T

C el/T(m

J/molK

2 )

T ( K )

∆0 / T c = 1 . 7 6 4 γ r e s + γ s T c a e x p ( - b T c / T )

γr e s

γnγs

( A ) ( B )

FIG. S2. Fully gapped bulk superconductivity revealby heat capacity measurement in Nb-doped Bi2Se3

(color online) (A) Sample heat capacity C is displayed as theratio of C and temperature T plotted against T . The zerofield curve is compared with the µ0H = 0.75 T curve. The0.75 T curve provides us a method to define the phonon con-tribution Cph. (B) The electronic part of heat capacity Cel is

shown as CelT

vs. T . A clear kink is observed at the super-conducting transition temperature Tc. Near T ∼ 0, the curveapproaches a finite value in Cel/T and gives a measurement ofthe non-superconducting volume fraction around 20% in thissample. As T increases from the base temperature, Cel/Tfollows the exponential curve(dashed pink line), as expectedfrom a fully gapped superconductor. Numerical calculationof Cel in BCS superconductors are shown with the only pa-rameter α ≡ kBTc

∆, where ∆ is the superconducting gap. For

the overall trace, α = 1.76 trace gives the best fit of the heatcapacity trace below Tc.

The exponential decay, rather than a typical power-law dependence, observed in Cel − T is indicative of anodeless superconducting gap. Other heat capacity pa-pers on unconventional superconductors fairly recognizethat a high order power-law can be hard to distinguishfrom an exponential decay at low temperatures [10]. Forline nodes, Cel

T should fall linearly with T , and for point

nodes, CelT should go as T 2 [4]. Line nodes, as have beenseen in other unconventional superconductors [5–7], arenot consistent with our data.

We note that CelT trace approaches a finite value γres

near the base temperature. This suggests a partial non-

superconducting volume in the crystal of about γresγn∼

<20%. This value is consistent with the fact that theMeissner effect shows non superconducting volume variesbetween 10% and 30% in different samples. On the otherhand, it is possible that special nodal gap structures maylead to residual density of states. Other experiments,such as the penetration depth experiment, would giveclear answers on whether the superconducting gap has anode.

( B )

0

3 0

6 09 0

1 2 0

1 5 0

1 8 0

2 1 0

2 4 02 7 0

3 0 0

3 3 0

0 . 00 . 51 . 01 . 52 . 02 . 53 . 0

0 . 00 . 51 . 01 . 52 . 02 . 53 . 0

∆τ =

τ+-τ

-(10-9 Nm

)

0 . 0 1 T 0 . 1 0 . 2 0 . 3 0 . 4 0 . 5

T = 0 . 3 K

0

3 0

6 09 0

1 2 0

1 5 0

1 8 0

2 1 0

2 4 02 7 0

3 0 0

3 3 0

0 . 00 . 51 . 01 . 52 . 02 . 5

0 . 00 . 51 . 01 . 52 . 02 . 5

T = 2 0 m K ∆τ

= τ

+-τ-(1

0-9 Nm)

0 . 0 1 T 0 . 1 0 . 2 0 . 3 0 . 4 0 . 5

( A )

FIG. S3. ∆τ = τ+ − τ− vs φ for two different cool downs.(color online) Polar plot of hysteresis loop magnitude ∆τ =τ+ − τ− for two different cool downs. τ+ is the torque signalfrom the up-sweep of the magnetic field, and τ− is the torquesignal from the down-sweep of the magnetic field. (A) Cooldown of Nb-doped Bi2Se3 to the 300 mK base temperaturein a He-3 cryostat. (B) Cool down of Nb-doped Bi2Se3 to the20 mK base temperature in a dilution refrigerator.

Symmetry Breaking in different cooling down

Fig.S3 shows ∆τ = τ+ − τ− vs φ for the same Nb-doped Bi2Se3 in two different cool downs. The data inPanel A if from a cool down in a He-3 cryostat, and panel(B) is from a different cool down in a dilution refrigerator.

S3

Due to the mechanical vibration in the dilution refriger-ator, the curve in Panel B is nosier. However, iIn bothcases, the preferred axis is locked on a crystalline mirrorplane. However, iIn the two cases, the preferred axis isalong roughly the same axis. This second cooling dataconfirms the observation of the spontaneous symmetrybreaking in Nb-doped Bi2Se3.

Rotational Symmetry Breaking in other Nb-dopedBi2Se3 samples

We repeated the search for rotational symmetry break-ing in the hysteretic property of Nb-doped Bi2Se3 with asecond piece of superconducting Nb-doped Bi2Se3. Thissample is designated sample E. Sample E was cooleddown to 20 mK in a dilution refrigerator. We swept thefield up and down from -1 T to 1 T to measure the entiresuperconducting hysteresis loop. As in the main text,we applied torque magnetometry to map the completeangular dependence of its in-plane magnetic anisotropy.

Fig.S4(A) shows the effective magnetization loop∆M = M+ −M− versus angle φ of sample E. There is aconstant background magnetization labeled A0. The ef-fective magnetization loop ∆M follows A0+A2φ sin(2φ−α) + A4φsin(4φ − β). This angular-independent offsetlikely arises from the torsional twist of the cantileversetup, although further experiments are needed to deter-mine the exact origin. For further analysis, we subtractaway the A0 term to get the angular dependence of thein-plane effective magnetization.

Fig.S4(B) shows a polar plot of the effective magne-tization versus angle φ of sample E. The superconduct-ing hysteresis loop closes at regular intervals correspond-ing to the axes normal to the mirror planes of the crys-tal structure. The rotational symmetry is again broken.This is consistent with the sample from the main text.We note that there is additional A2φ contribution, whichprobably originates from a small misaligning the mag-netic field slightly away from the crystalline ab plane.(The misalignment is about 2◦, estimated by comparing

the susceptibility anisotropy in this experiment with thatin a regular torque measurement with magnetic field ro-tating in the crystalline c and a axis.) However, A4φ

contribution could be still observed. The consistency ofthe spontaneous symmetry breaking between different su-perconducting Nb-doped Bi2Se3 samples suggests this isa intrinsic feature of the system and not the result ofgeometric anisotropy in any given sample.

Fig.S4(C) shows a FFT showing the relative strengthof the nematic term, sin(2φ), and the crystal symme-try term, sin(4φ). The first peak, A2φ, is the amplitudeof the nematic order sin(2φ). The second peak, A4φ,

in the amplitude of the sin(4φ) term. The ratioA2φ

A4φis

a measure of the strength of the nematic order over thebackground crystal symmetry. As seen in Fig.S4(D), A2φ

is the leading term near zero field. When H increasesclose to the upper critical field, A2φ dramatically van-ishes. The vanishing of the nematic order as the super-conductor approaches the normal state follows the sametrend as the sample shown in the main text. Further-more, this behavior in the superconducting state is verydifferent from the angular dependence of the magneticsusceptibility in the normal state above Hc2. Fig. S4(E)shows the normal state effective magnetic susceptibility,which displays roughly a 60◦ period, confirming the crys-talline symmetry in the normal state.

[1] C. P. Bean, Phys. Rev. Lett. 8, 250 (1962).[2] C. P. Bean, Rev. Mod. Phys 36, 31 (1964).[3] Lu Li, ”Torque Magnetometry in Unconventional Super-

conductors”, Ph.D. Thesis, Princeton University (2008).[4] M. Sigrist and K. Ueda, Rev. Mod. Phys. 63, 239 (1991).[5] Z. Q. Mao, Y. Maeno, S. NishiZaki, T. Akima, and T.

Ishiguro, Phys. Rev. Lett. 84, 991 (2000).[6] K. Hasselbach, J. R. Kirtley, J. Flouquet, Phys. Rev. B

47, 509 (1993).[7] R. Movshovich, M. Jaime, J. D. Thompson, C. Petrovic,

Z. Fisk, P. G. Pagliuso, and J. L. Sarrao, Phys. Rev. Lett.86, 5152 (2001).

S4

0

3 0

6 09 0

1 2 0

1 5 0

1 8 0

2 1 0

2 4 02 7 0

3 0 0

3 3 0

0

1 0

2 0

0

1 0

2 0

T = 2 0 m K

∆Meff

= M+-M

- (10-9 J/

T)

0 . 0 3 T 0 . 1 0 . 2 0 . 3 0 . 4 0 . 5

0 6 0 1 2 0 1 8 0 2 4 0 3 0 0 3 6 0 4 2 0- 3 0- 2 5- 2 0- 1 5- 1 0- 505

1 0 S a m p l e EEff

ective

mag

netiza

tion ∆

M eff (1

0-9 J/T)

φ ( d e g r e e )

0 . 0 3 T - 0 . 0 3

A 0

0 5 1 00 . 00 . 20 . 40 . 60 . 81 . 01 . 2

A 4 φ

A 2 φ

FFT A

mplitu

de (a

.u.)

F r e q u e n c y

B = 0 . 0 3 TT = 2 0 m K

0 5 0 1 0 0 1 5 0 2 0 0 2 5 0 3 0 0 3 5 0 4 0 0

- 3

- 2

- 1

0

1

χ = dM

/dH (1

0-9 Nm/T2 )

φ ( d e g r e e )

0 . 9 T - 0 . 9

S a m p l e E

( D )

( C )

( E )

( B )

0 . 0 0 . 1 0 . 2 0 . 3 0 . 4 0 . 5 0 . 6

1 E - 4

0 . 0 0 1 A 2 φ

A 4 φ

T = 2 0 m K

FFT a

mplitu

de (a

.u.)

µ0 H ( T )

( A )

FIG. S4. Rotational Symmetry Breaking in Sample E (color online) (A) The effective magnetization loop ∆M = M+−M−versus angle φ for sample E. The red baseline marks a constant offset in the signal that was subtracted out. (B) Polar plotof the effective magnetization in sample E. (C) Here is a Fast Fourier Transform showing the relative strength of the nematicterm, sin 2φ, and the crystalline symmetric term, sin(4φ). (D) Ratio of the FFT amplitudes A2φ and A4φ against externalmagnetic field in the superconducting state of sample E. Above the 0.6 T, the superconducting hysteresis loop quickly vanishes.(E) Normal state susceptibility from sample E at 0.9 T, indicating sin 6φ behavior.