summaryrefslogtreecommitdiff
path: root/test/Transforms/SLPVectorizer
Commit message (Collapse)AuthorAge
* Reduce verbiage of lit.local.cfg filesAlp Toker2014-06-09
| | | | | | We can just split targets_to_build in one place and make it immutable. git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@210496 91177308-0d34-0410-b5e6-96231b3b80d8
* Fix typo in a test from r210342.Michael Zolotukhin2014-06-06
| | | | git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@210343 91177308-0d34-0410-b5e6-96231b3b80d8
* [SLP] Enable vectorization of GEP expressions.Michael Zolotukhin2014-06-06
| | | | | | | | The use cases look like the following: x->a = y->a + 10 x->b = y->b + 12 git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@210342 91177308-0d34-0410-b5e6-96231b3b80d8
* Fix PR19657 (scalar loads not combined into vector load)Karthik Bhat2014-06-06
| | | | | | | | | | If we have common uses on separate paths in the tree; process the one with greater common depth first. This makes sure that we do not assume we need to extract a load when it is actually going to be part of a vectorized tree. Review: http://reviews.llvm.org/D3800 git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@210310 91177308-0d34-0410-b5e6-96231b3b80d8
* Allow vectorization of intrinsics such as powi,cttz and ctlz in Loop and SLP ↵Karthik Bhat2014-05-30
| | | | | | | | | | | Vectorizer. This patch adds support to vectorize intrinsics such as powi, cttz and ctlz in Vectorizer. These intrinsics are different from other intrinsics as second argument to these function must be same in order to vectorize them and it should be represented as a scalar. Review: http://reviews.llvm.org/D3851#inline-32769 and http://reviews.llvm.org/D3937#inline-32857 git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@209873 91177308-0d34-0410-b5e6-96231b3b80d8
* AArch64/ARM64: move ARM64 into AArch64's placeTim Northover2014-05-24
| | | | | | | | | | | | | | | This commit starts with a "git mv ARM64 AArch64" and continues out from there, renaming the C++ classes, intrinsics, and other target-local objects for consistency. "ARM64" test directories are also moved, and tests that began their life in ARM64 use an arm64 triple, those from AArch64 use an aarch64 triple. Both should be equivalent though. This finishes the AArch64 merge, and everyone should feel free to continue committing as normal now. git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@209577 91177308-0d34-0410-b5e6-96231b3b80d8
* Implement sext(C1 + C2*X) --> sext(C1) + sext(C2*X) andMichael Zolotukhin2014-05-24
| | | | | | | | | | | sext{C1,+,C2} --> sext(C1) + sext{0,+,C2} transformation in Scalar Evolution. That helps SLP-vectorizer to recognize consecutive loads/stores. <rdar://problem/14860614> git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@209568 91177308-0d34-0410-b5e6-96231b3b80d8
* Fix broken FileCheck prefixesNico Rieck2014-05-23
| | | | git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@209538 91177308-0d34-0410-b5e6-96231b3b80d8
* Flip on vectorization of bswap intrinsics.Benjamin Kramer2014-05-19
| | | | | | | | | The cost model conservatively assumes that it will always get scalarized and that's about as good as we can get with the generic TTI; reasoning whether a shuffle with an efficient lowering is available is hard. We can override that conservative estimate for some targets in the future. git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@209125 91177308-0d34-0410-b5e6-96231b3b80d8
* SLPVectorizer: When sorting by domination for CSE don't assert on ↵Benjamin Kramer2014-05-09
| | | | | | | | | | | | unreachable code. There is no total ordering if the CFG is disconnected. We don't care if we catch all CSE opportunities in dead code either so just exclude ignore them in the assert. PR19646 git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@208461 91177308-0d34-0410-b5e6-96231b3b80d8
* Always set alignment of vectorized LD/ST in SLP-Vectorizer. ↵Yi Jiang2014-05-05
| | | | | | <rdar://problem/16812145> git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@207983 91177308-0d34-0410-b5e6-96231b3b80d8
* SLPVectorizer: Bring back the insertelement patch (r205965) with fixesArnold Schwaighofer2014-05-04
| | | | | | | | | | | | | | | | When can't assume a vectorized tree is rooted in an instruction. The IRBuilder could have constant folded it. When we rebuild the build_vector (the series of InsertElement instructions) use the last original InsertElement instruction. The vectorized tree root is guaranteed to be before it. Also, we can't assume that the n-th InsertElement inserts the n-th element into a vector. This reverts r207746 which reverted the revert of the revert of r205018 or so. Fixes the test case in PR19621. git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@207939 91177308-0d34-0410-b5e6-96231b3b80d8
* Vectorize intrinsic math function calls in SLPVectorizer.Karthik Bhat2014-05-03
| | | | | | | | This patch adds support to recognize and vectorize intrinsic math functions in SLPVectorizer. Review: http://reviews.llvm.org/D3560 and http://reviews.llvm.org/D3559 git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@207901 91177308-0d34-0410-b5e6-96231b3b80d8
* Revert r205965, which essentially reverts r205018 for the second time.Chandler Carruth2014-05-01
| | | | | | | | | | | | | | | | =[ Turns out that this was the root cause of PR19621. We found a crasher only recently (likely due to improvements elsewhere in the SLP vectorizer) but the reduced test case failed all the way back to here. I've confirmed that reverting this patch both fixes the reduced test case in PR19621 and the actual source file that led to it, so it seems to really be rooted here. I've replied to the commit thread with discussion of my (feeble) attempts to debug this. Didn't make it very far, so reverting now that we have a good test case so that things can get back to healthy while the debugging carries on. git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@207746 91177308-0d34-0410-b5e6-96231b3b80d8
* Continue slp vectorization even the BB already has vectorized store ↵Yi Jiang2014-04-29
| | | | | | radar://16641956 git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@207572 91177308-0d34-0410-b5e6-96231b3b80d8
* Reapply "SLPVectorizer: Ignore users that are insertelements we can ↵Arnold Schwaighofer2014-04-10
| | | | | | | | | reschedule them" This commit reapplies 205018. After 205855 we should correctly vectorize intrinsics. git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@205965 91177308-0d34-0410-b5e6-96231b3b80d8
* SLPVectorizer: Only vectorize intrinsics whose operands are widened equallyArnold Schwaighofer2014-04-09
| | | | | | | | | The vectorizer only knows how to vectorize intrinics by widening all operands by the same factor. Patch by Tyler Nowicki! git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@205855 91177308-0d34-0410-b5e6-96231b3b80d8
* SLPVectorizer: compare entire intrinsic for SLP compatibility.Tim Northover2014-04-02
| | | | | | | | | Some Intrinsics are overloaded to the extent that return type equality (all that's been checked up to now) does not guarantee that the arguments are the same. In these cases SLP vectorizer should not recurse into the operands, which can be achieved by comparing them as "Function *" rather than simply the ID. git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@205424 91177308-0d34-0410-b5e6-96231b3b80d8
* Revert "SLPVectorizer: Ignore users that are insertelements we can ↵Arnold Schwaighofer2014-03-31
| | | | | | | | | | | | | | reschedule them" This reverts commit r205018. Conflicts: lib/Transforms/Vectorize/SLPVectorizer.cpp test/Transforms/SLPVectorizer/X86/insert-element-build-vector.ll This is breaking libclc build. git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@205260 91177308-0d34-0410-b5e6-96231b3b80d8
* SLPVectorizer: Take credit for free extractelement instructionsArnold Schwaighofer2014-03-28
| | | | | | | | | Extract element instructions that will be removed when vectorzing lower the cost. Patch by Arch D. Robison! git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@205020 91177308-0d34-0410-b5e6-96231b3b80d8
* SLPVectorizer: Ignore users that are insertelements we can reschedule themArnold Schwaighofer2014-03-28
| | | | | | Patch by Arch D. Robison! git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@205018 91177308-0d34-0410-b5e6-96231b3b80d8
* Resubmit "[SLPV] Recognize vectorizable intrinsics during SLP vectorization ..."Raul E. Silvera2014-03-12
| | | | | | | This reverts commit 86cb795388643710dab34941ddcb5a9470ac39d8. The problems previously found have been resolved through other CLs. git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@203707 91177308-0d34-0410-b5e6-96231b3b80d8
* Trivial test commit.Raul E. Silvera2014-03-05
| | | | git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@202924 91177308-0d34-0410-b5e6-96231b3b80d8
* SLPVectorizer: Try vectorizing 'splat' storesArnold Schwaighofer2014-02-24
| | | | | | | | | Vectorize sequential stores of a broadcasted value. 5% on eon. radar://16124699 git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@202067 91177308-0d34-0410-b5e6-96231b3b80d8
* fix for null VectorizedValue assertion in the SLP Vectorizer (in function ↵Gerolf Hoflehner2014-02-17
| | | | | | vectorizeTree()). radar://16064178 git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@201501 91177308-0d34-0410-b5e6-96231b3b80d8
* Revert "[SLPV] Recognize vectorizable intrinsics during SLP vectorization ..."Reid Kleckner2014-02-01
| | | | | | | This reverts commit r200576. It broke 32-bit self-host builds by vectorizing two calls to @llvm.bswap.i64, which we then fail to expand. git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@200602 91177308-0d34-0410-b5e6-96231b3b80d8
* [SLPV] Recognize vectorizable intrinsics during SLP vectorization andChandler Carruth2014-01-31
| | | | | | | | | | transform accordingly. Based on similar code from Loop vectorization. Subsequent commits will include vectorization of function calls to vector intrinsics and form function calls to vector library calls. Patch by Raul Silvera! (Much delayed due to my not running dcommit) git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@200576 91177308-0d34-0410-b5e6-96231b3b80d8
* Fix broken CHECK lines.Benjamin Kramer2014-01-11
| | | | git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@199016 91177308-0d34-0410-b5e6-96231b3b80d8
* SLPVectorizer: An in-tree vectorized entry cannot also be a scalar external useArnold Schwaighofer2013-12-05
| | | | | | | | | | | | | | | We were creating external uses for scalar values in MustGather entries that also had a ScalarToTreeEntry (they also are present in a vectorized tuple). This meant we would keep a value 'alive' as a scalar and vectorized causing havoc. This is not necessary because when we create a MustGather vector we explicitly create external uses entries for the insertelement instructions of the MustGather vector elements. Fixes PR18129. radar://15582184 git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@196508 91177308-0d34-0410-b5e6-96231b3b80d8
* opt: Mirror vectorization presets of clangArnold Schwaighofer2013-12-03
| | | | | | | | | | clang enables vectorization at optimization levels > 1 and size level < 2. opt should behave similarily. Loop vectorization and SLP vectorization can be disabled with the flags -disable-(loop/slp)-vectorization. git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@196294 91177308-0d34-0410-b5e6-96231b3b80d8
* PR1860 - We can't save a list of ExtractElement instructions to CSE because ↵Nadav Rotem2013-11-26
| | | | | | | | | | some of these instructions may be removed and optimized in future iterations. Instead we save a list of basic blocks that we need to CSE. git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@195791 91177308-0d34-0410-b5e6-96231b3b80d8
* PR18060 - When we RAUW values with ExtractElement instructions in some casesNadav Rotem2013-11-26
| | | | | | | | | | we generate PHI nodes with multiple entries from the same basic block but with different values. Enabling CSE on ExtractElement instructions make sure that all of the RAUWed instructions are the same. git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@195773 91177308-0d34-0410-b5e6-96231b3b80d8
* Add the test case that I missed when committing r195528. Doh!Chandler Carruth2013-11-25
| | | | git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@195691 91177308-0d34-0410-b5e6-96231b3b80d8
* Debug Info: update testing cases to specify the debug info version number.Manman Ren2013-11-22
| | | | | | | | | We are going to drop debug info without a version number or with a different version number, to make sure we don't crash when we see bitcode files with different debug info metadata format. git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@195504 91177308-0d34-0410-b5e6-96231b3b80d8
* SLP Vectorizer: Extract cost will only be added once even if the scalar has ↵Yi Jiang2013-11-22
| | | | | | multiple external uses. git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@195406 91177308-0d34-0410-b5e6-96231b3b80d8
* Add llvm/test/Transforms/SLPVectorizer/ARM/lit.local.cfg. Tests there ↵NAKAMURA Takumi2013-10-29
| | | | | | require ARM in targets. git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@193580 91177308-0d34-0410-b5e6-96231b3b80d8
* ARM cost model: Unaligned vectorized double stores are expensiveArnold Schwaighofer2013-10-29
| | | | | | | | | Updated a test case that assumed that <2 x double> would vectorize to use <4 x float>. radar://15338229 git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@193574 91177308-0d34-0410-b5e6-96231b3b80d8
* ARM cost model: Account for zero cost scalar SROA instructionsArnold Schwaighofer2013-10-29
| | | | | | | | | By vectorizing a series of srl, or, ... instructions we have obfuscated the intention so much that the backend does not know how to fold this code away. radar://15336950 git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@193573 91177308-0d34-0410-b5e6-96231b3b80d8
* SLPVectorizer: Don't vectorize volatile memory operationsArnold Schwaighofer2013-10-16
| | | | | | | | | | radar://15231682 Reapply r192799, http://lab.llvm.org:8011/builders/lldb-x86_64-debian-clang/builds/8226 showed that the bot is still broken even with this out. git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@192820 91177308-0d34-0410-b5e6-96231b3b80d8
* Revert "SLPVectorizer: Don't vectorize volatile memory operations"Arnold Schwaighofer2013-10-16
| | | | | | This speculatively reverts commit 192799. It might have broken a linux buildbot. git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@192816 91177308-0d34-0410-b5e6-96231b3b80d8
* SLPVectorizer: Don't vectorize volatile memory operationsArnold Schwaighofer2013-10-16
| | | | | | radar://15231682 git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@192799 91177308-0d34-0410-b5e6-96231b3b80d8
* SLPVectorizer: Sort PHINodes based on their opcodeArnold Schwaighofer2013-10-12
| | | | | | | | | | | | | | Before this patch we relied on the order of phi nodes when we looked for phi nodes of the same type. This could prevent vectorization of cases where there was a phi node of a second type in between phi nodes of some type. This is important for vectorization of an internal graphics kernel. On the test suite + external on x86_64 (and on a run on armv7s) it showed no impact on either performance or compile time. radar://15024459 git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@192537 91177308-0d34-0410-b5e6-96231b3b80d8
* SLPVectorizer: Sort inputs to commutative binary operationsArnold Schwaighofer2013-10-04
| | | | | | | | | | | | | | | | | | | | | | | Sort the operands of the other entries in the current vectorization root according to the first entry's operands opcodes. %conv0 = uitofp ... %load0 = load float ... = fmul %conv0, %load0 = fmul %load0, %conv1 = fmul %load0, %conv2 Make sure that we recursively vectorize <%conv0, %conv1, %conv2> and <%load0, %load0, %load0>. This makes it more likely to obtain vectorizable trees. We have to be careful when we sort that we don't destroy 'good' existing ordering implied by source order. radar://15080067 git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@191977 91177308-0d34-0410-b5e6-96231b3b80d8
* Apply slp vectorization on fully-vectorizable tree of height 2Yi Jiang2013-10-02
| | | | git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@191852 91177308-0d34-0410-b5e6-96231b3b80d8
* SLPVectorizer: Make store chain finding more aggressive with ↵Benjamin Kramer2013-10-02
| | | | | | | | | GetUnderlyingObject. This recursively strips all GEPs like the existing code. It also handles bitcasts and other operations that do not change the pointer value. git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@191847 91177308-0d34-0410-b5e6-96231b3b80d8
* TBAA: update tbaa format from scalar format to struct-path aware format.Manman Ren2013-09-30
| | | | git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@191690 91177308-0d34-0410-b5e6-96231b3b80d8
* TBAA: remove !tbaa from testing cases when they are not needed.Manman Ren2013-09-30
| | | | git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@191689 91177308-0d34-0410-b5e6-96231b3b80d8
* IRBuilder: Add RAII objects to reset insertion points or fast math flags.Benjamin Kramer2013-09-30
| | | | | | | | Inspired by the object from the SLPVectorizer. This found a minor bug in the debug loc restoration in the vectorizer where the location of a following instruction was attached instead of the location from the original instruction. git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@191673 91177308-0d34-0410-b5e6-96231b3b80d8
* Fix SLPVectorizer using wrong address space for load/storeMatt Arsenault2013-09-27
| | | | git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@191564 91177308-0d34-0410-b5e6-96231b3b80d8
* Transforms: Use getFirstNonPHI to set the insertion point for PHIsJustin Bogner2013-09-27
| | | | | | | | | | We were previously using getFirstInsertionPt to insert PHI instructions when vectorizing, but getFirstInsertionPt also skips past landingpads, causing this to generate invalid IR. We can avoid this issue by using getFirstNonPHI instead. git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@191526 91177308-0d34-0410-b5e6-96231b3b80d8
generated by cgit v1.2.3 (git 2.25.1) at 2024-06-14 15:40:32 +0000