summaryrefslogtreecommitdiff
path: root/test/Transforms/SLPVectorizer
Commit message (Collapse)AuthorAge
* opt: Mirror vectorization presets of clangArnold Schwaighofer2013-12-03
| | | | | | | | | | clang enables vectorization at optimization levels > 1 and size level < 2. opt should behave similarily. Loop vectorization and SLP vectorization can be disabled with the flags -disable-(loop/slp)-vectorization. git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@196294 91177308-0d34-0410-b5e6-96231b3b80d8
* PR1860 - We can't save a list of ExtractElement instructions to CSE because ↵Nadav Rotem2013-11-26
| | | | | | | | | | some of these instructions may be removed and optimized in future iterations. Instead we save a list of basic blocks that we need to CSE. git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@195791 91177308-0d34-0410-b5e6-96231b3b80d8
* PR18060 - When we RAUW values with ExtractElement instructions in some casesNadav Rotem2013-11-26
| | | | | | | | | | we generate PHI nodes with multiple entries from the same basic block but with different values. Enabling CSE on ExtractElement instructions make sure that all of the RAUWed instructions are the same. git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@195773 91177308-0d34-0410-b5e6-96231b3b80d8
* Add the test case that I missed when committing r195528. Doh!Chandler Carruth2013-11-25
| | | | git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@195691 91177308-0d34-0410-b5e6-96231b3b80d8
* Debug Info: update testing cases to specify the debug info version number.Manman Ren2013-11-22
| | | | | | | | | We are going to drop debug info without a version number or with a different version number, to make sure we don't crash when we see bitcode files with different debug info metadata format. git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@195504 91177308-0d34-0410-b5e6-96231b3b80d8
* SLP Vectorizer: Extract cost will only be added once even if the scalar has ↵Yi Jiang2013-11-22
| | | | | | multiple external uses. git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@195406 91177308-0d34-0410-b5e6-96231b3b80d8
* Add llvm/test/Transforms/SLPVectorizer/ARM/lit.local.cfg. Tests there ↵NAKAMURA Takumi2013-10-29
| | | | | | require ARM in targets. git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@193580 91177308-0d34-0410-b5e6-96231b3b80d8
* ARM cost model: Unaligned vectorized double stores are expensiveArnold Schwaighofer2013-10-29
| | | | | | | | | Updated a test case that assumed that <2 x double> would vectorize to use <4 x float>. radar://15338229 git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@193574 91177308-0d34-0410-b5e6-96231b3b80d8
* ARM cost model: Account for zero cost scalar SROA instructionsArnold Schwaighofer2013-10-29
| | | | | | | | | By vectorizing a series of srl, or, ... instructions we have obfuscated the intention so much that the backend does not know how to fold this code away. radar://15336950 git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@193573 91177308-0d34-0410-b5e6-96231b3b80d8
* SLPVectorizer: Don't vectorize volatile memory operationsArnold Schwaighofer2013-10-16
| | | | | | | | | | radar://15231682 Reapply r192799, http://lab.llvm.org:8011/builders/lldb-x86_64-debian-clang/builds/8226 showed that the bot is still broken even with this out. git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@192820 91177308-0d34-0410-b5e6-96231b3b80d8
* Revert "SLPVectorizer: Don't vectorize volatile memory operations"Arnold Schwaighofer2013-10-16
| | | | | | This speculatively reverts commit 192799. It might have broken a linux buildbot. git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@192816 91177308-0d34-0410-b5e6-96231b3b80d8
* SLPVectorizer: Don't vectorize volatile memory operationsArnold Schwaighofer2013-10-16
| | | | | | radar://15231682 git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@192799 91177308-0d34-0410-b5e6-96231b3b80d8
* SLPVectorizer: Sort PHINodes based on their opcodeArnold Schwaighofer2013-10-12
| | | | | | | | | | | | | | Before this patch we relied on the order of phi nodes when we looked for phi nodes of the same type. This could prevent vectorization of cases where there was a phi node of a second type in between phi nodes of some type. This is important for vectorization of an internal graphics kernel. On the test suite + external on x86_64 (and on a run on armv7s) it showed no impact on either performance or compile time. radar://15024459 git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@192537 91177308-0d34-0410-b5e6-96231b3b80d8
* SLPVectorizer: Sort inputs to commutative binary operationsArnold Schwaighofer2013-10-04
| | | | | | | | | | | | | | | | | | | | | | | Sort the operands of the other entries in the current vectorization root according to the first entry's operands opcodes. %conv0 = uitofp ... %load0 = load float ... = fmul %conv0, %load0 = fmul %load0, %conv1 = fmul %load0, %conv2 Make sure that we recursively vectorize <%conv0, %conv1, %conv2> and <%load0, %load0, %load0>. This makes it more likely to obtain vectorizable trees. We have to be careful when we sort that we don't destroy 'good' existing ordering implied by source order. radar://15080067 git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@191977 91177308-0d34-0410-b5e6-96231b3b80d8
* Apply slp vectorization on fully-vectorizable tree of height 2Yi Jiang2013-10-02
| | | | git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@191852 91177308-0d34-0410-b5e6-96231b3b80d8
* SLPVectorizer: Make store chain finding more aggressive with ↵Benjamin Kramer2013-10-02
| | | | | | | | | GetUnderlyingObject. This recursively strips all GEPs like the existing code. It also handles bitcasts and other operations that do not change the pointer value. git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@191847 91177308-0d34-0410-b5e6-96231b3b80d8
* TBAA: update tbaa format from scalar format to struct-path aware format.Manman Ren2013-09-30
| | | | git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@191690 91177308-0d34-0410-b5e6-96231b3b80d8
* TBAA: remove !tbaa from testing cases when they are not needed.Manman Ren2013-09-30
| | | | git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@191689 91177308-0d34-0410-b5e6-96231b3b80d8
* IRBuilder: Add RAII objects to reset insertion points or fast math flags.Benjamin Kramer2013-09-30
| | | | | | | | Inspired by the object from the SLPVectorizer. This found a minor bug in the debug loc restoration in the vectorizer where the location of a following instruction was attached instead of the location from the original instruction. git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@191673 91177308-0d34-0410-b5e6-96231b3b80d8
* Fix SLPVectorizer using wrong address space for load/storeMatt Arsenault2013-09-27
| | | | git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@191564 91177308-0d34-0410-b5e6-96231b3b80d8
* Transforms: Use getFirstNonPHI to set the insertion point for PHIsJustin Bogner2013-09-27
| | | | | | | | | | We were previously using getFirstInsertionPt to insert PHI instructions when vectorizing, but getFirstInsertionPt also skips past landingpads, causing this to generate invalid IR. We can avoid this issue by using getFirstNonPHI instead. git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@191526 91177308-0d34-0410-b5e6-96231b3b80d8
* SLPVectorize: Put horizontal reductions feeding a store under separate flagArnold Schwaighofer2013-09-25
| | | | | | | Put them under a separate flag for experimentation. They are more likely to interfere with loop vectorization which happens later in the pass pipeline. git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@191371 91177308-0d34-0410-b5e6-96231b3b80d8
* Test case for r191314. Yi Jiang2013-09-24
| | | | | | | | Some supplemental information for r191314: We would like to make sure SLP Vectorizer will not try to vectorize tiny trees even with a negative threshold so we set the cost to INT_MAX. git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@191327 91177308-0d34-0410-b5e6-96231b3b80d8
* Reapply "SLPVectorizer: Handle more horizontal reductions (disabled)""Arnold Schwaighofer2013-09-21
| | | | | | | | | | | | | | Reapply r191108 with a fix for a memory corruption error I introduced. Of course, we can't reference the scalars that we replace by vectorizing and then call their eraseFromParent method. I only 'needed' the scalars to get the DebugLoc. Just store the DebugLoc before actually vectorizing instead. As a nice side effect, this also simplifies the interface between BoUpSLP and the HorizontalReduction class to returning a value pointer (the vectorized tree root). radar://14607682 git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@191123 91177308-0d34-0410-b5e6-96231b3b80d8
* Revert "SLPVectorizer: Handle more horizontal reductions (disabled)"Arnold Schwaighofer2013-09-21
| | | | | | | | | This reverts commit r191108. The horizontal.ll test case fails under libgmalloc. Thanks Shuxin for pointing this out to me. git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@191121 91177308-0d34-0410-b5e6-96231b3b80d8
* SLPVectorizer: Handle more horizontal reductions (disabled)Arnold Schwaighofer2013-09-20
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Match reductions starting at binary operation feeding into a phi. The code handles trees like r += v1 + v2 + v3 ... and r += v1 r += v2 ... and r *= v1 + v2 + ... We currently only handle associative operations (add, fadd fast). The code can now also handle reductions feeding into stores. a[i] = v1 + v2 + v3 + ... The code is currently disabled behind the flag "-slp-vectorize-hor". The cost model for most architectures is not there yet. I found one opportunity of a horizontal reduction feeding a phi in TSVC (LoopRerolling-flt) and there are several opportunities where reductions feed into stores. radar://14607682 git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@191108 91177308-0d34-0410-b5e6-96231b3b80d8
* Name the XCore target-specific subdirectories canonically.Chandler Carruth2013-09-18
| | | | git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@190940 91177308-0d34-0410-b5e6-96231b3b80d8
* A couple of tests, in llvm/test/Transforms/*/xcore, are XCore-specific. They ↵NAKAMURA Takumi2013-09-18
| | | | | | should be excluded when XCore is not built. git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@190938 91177308-0d34-0410-b5e6-96231b3b80d8
* Prevent LoopVectorizer and SLPVectorizer running if the target has no vector ↵Robert Lytton2013-09-18
| | | | | | | | | | registers. XCore target: Add XCoreTargetTransformInfo This is where getNumberOfRegisters() resides, which in turn returns the number of vector registers (=0). git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@190936 91177308-0d34-0410-b5e6-96231b3b80d8
* SLPVectorizer: Don't vectorize phi nodes that use invoke valuesArnold Schwaighofer2013-09-17
| | | | | | | | | We can't insert an insertelement after an invoke. We would have to split a critical edge. So when we see a phi node that uses an invoke we just give up. radar://14990770 git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@190871 91177308-0d34-0410-b5e6-96231b3b80d8
* Debug Info Testing: updated to use NULL instead of "i32 0" in a few fields.Manman Ren2013-09-06
| | | | | | | | | Field 2 of DIType (Context), field 9 of DIDerivedType (TypeDerivedFrom), field 12 of DICompositeType (ContainingType), fields 2, 7, 12 of DISubprogram (Context, Type, ContainingType). git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@190205 91177308-0d34-0410-b5e6-96231b3b80d8
* In this patch we are trying to do two things:Yi Jiang2013-09-03
| | | | | | | | | 1) If the width of vectorization list candidate is bigger than vector reg width, we will break it down to fit the vector reg. 2) We do not vectorize the width which is not power of two. The performance result shows it will help some spec benchmarks. mesa improved 6.97% and ammp improved 1.54%. git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@189830 91177308-0d34-0410-b5e6-96231b3b80d8
* Fix inserting instructions before last in bundle.Matt Arsenault2013-08-26
| | | | | | | | | | | The builder inserts from before the insert point, not after, so this would insert before the last instruction in the bundle instead of after it. I'm not sure if this can actually be a problem with any of the current insertions. git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@189285 91177308-0d34-0410-b5e6-96231b3b80d8
* Debug Info: add an identifier field to DICompositeType.Manman Ren2013-08-26
| | | | | | | | | | | | | | | | | | | DICompositeType will have an identifier field at position 14. For now, the field is set to null in DIBuilder. For DICompositeTypes where the template argument field (the 13th field) was optional, modify DIBuilder to make sure the template argument field is set. Now DICompositeType has 15 fields. Update DIBuilder to use NULL instead of "i32 0" for null value of a MDNode. Update verifier to check that DICompositeType has 15 fields and the last field is null or a MDString. Update testing cases to include an extra field for DICompositeType. The identifier field will be used by type uniquing so a front end can genearte a DICompositeType with a unique identifer. git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@189282 91177308-0d34-0410-b5e6-96231b3b80d8
* Forgot to add slp threshold to testMatt Arsenault2013-08-26
| | | | git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@189248 91177308-0d34-0410-b5e6-96231b3b80d8
* Vectorize starting from insertelements building a vectorMatt Arsenault2013-08-26
| | | | git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@189233 91177308-0d34-0410-b5e6-96231b3b80d8
* [Debug Info Tests] Update testing cases.Manman Ren2013-08-22
| | | | | | | | | | A single metadata will not span multiple lines. This also helps me with my script to automatic update the testing cases. A debug info testing case should have a llvm.dbg.cu. Do not use hard-coded id for debug nodes. git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@189033 91177308-0d34-0410-b5e6-96231b3b80d8
* Teach the SLP vectorizer the correct way to check for consecutive accessChandler Carruth2013-08-22
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | using GEPs. Previously, it used a number of different heuristics for analyzing the GEPs. Several of these were conservatively correct, but failed to fall back to SCEV even when SCEV might have given a reasonable answer. One was simply incorrect in how it was formulated. There was good code already to recursively evaluate the constant offsets in GEPs, look through pointer casts, etc. I gathered this into a form code like the SLP code can use in a previous commit, which allows all of this code to become quite simple. There is some performance (compile time) concern here at first glance as we're directly attempting to walk both pointers constant GEP chains. However, a couple of thoughts: 1) The very common cases where there is a dynamic pointer, and a second pointer at a constant offset (usually a stride) from it, this code will actually not do any unnecessary work. 2) InstCombine and other passes work very hard to collapse constant GEPs, so it will be rare that we iterate here for a long time. That said, if there remain performance problems here, there are some obvious things that can improve the situation immensely. Doing a vectorizer-pass-wide memoizer for each individual layer of pointer values, their base values, and the constant offset is likely to be able to completely remove redundant work and strictly limit the scaling of the work to scrape these GEPs. Since this optimization was not done on the prior version (which would still benefit from it), I've not done it here. But if folks have benchmarks that slow down it should be straight forward for them to add. I've added a test case, but I'm not really confident of the amount of testing done for different access patterns, strides, and pointer manipulation. git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@189007 91177308-0d34-0410-b5e6-96231b3b80d8
* SLPVectorizer: Fix invalid iterator errorsArnold Schwaighofer2013-08-20
| | | | | | | | | | | Update iterator when the SLP vectorizer changes the instructions in the basic block by restarting the traversal of the basic block. Patch by Yi Jiang! Fixes PR 16899. git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@188832 91177308-0d34-0410-b5e6-96231b3b80d8
* [tests] Cleanup initialization of test suffixes.Daniel Dunbar2013-08-16
| | | | | | | | | | | | | | | | | - Instead of setting the suffixes in a bunch of places, just set one master list in the top-level config. We now only modify the suffix list in a few suites that have one particular unique suffix (.ml, .mc, .yaml, .td, .py). - Aside from removing the need for a bunch of lit.local.cfg files, this enables 4 tests that were inadvertently being skipped (one in Transforms/BranchFolding, a .s file each in DebugInfo/AArch64 and CodeGen/PowerPC, and one in CodeGen/SI which is now failing and has been XFAILED). - This commit also fixes a bunch of config files to use config.root instead of older copy-pasted code. git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@188513 91177308-0d34-0410-b5e6-96231b3b80d8
* Fix PR16797 - Support PHINodes with multiple inputs from the same basic block.Nadav Rotem2013-08-12
| | | | | | | | | Do not generate new vector values for the same entries because we know that the incoming values from the same block must be identical. git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@188185 91177308-0d34-0410-b5e6-96231b3b80d8
* SLPVectorizer: Fix PR16777. PHInodes may use multiple extracted values that ↵Nadav Rotem2013-08-02
| | | | | | | | | | come from different blocks. Thanks Alexey Samsonov. git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@187663 91177308-0d34-0410-b5e6-96231b3b80d8
* Add the C source code to the test to make it easier to update when debug ↵Nadav Rotem2013-07-29
| | | | | | | | | | info changes. Thanks Eric. git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@187368 91177308-0d34-0410-b5e6-96231b3b80d8
* SLPVectorier: update the debug location for the new instructions.Nadav Rotem2013-07-29
| | | | git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@187363 91177308-0d34-0410-b5e6-96231b3b80d8
* Don't vectorize when the attribute NoImplicitFloat is used.Nadav Rotem2013-07-29
| | | | git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@187340 91177308-0d34-0410-b5e6-96231b3b80d8
* SLP Vectorier: Don't vectorize really short chains because they are already ↵Nadav Rotem2013-07-26
| | | | | | handled by the SelectionDAG store-vectorizer, which does a better job in deciding when to vectorize. git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@187267 91177308-0d34-0410-b5e6-96231b3b80d8
* SLP Vectorizer: Disable the vectorization of non power of two chains, such ↵Nadav Rotem2013-07-26
| | | | | | as <3 x float>, because we dont have a good cost model for these types. git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@187265 91177308-0d34-0410-b5e6-96231b3b80d8
* When we vectorize across multiple basic blocks we may vectorize PHINodes ↵Nadav Rotem2013-07-22
| | | | | | that create a cycle. We already break the cycle on phi-nodes, but arithmetic operations are still uplicated. This patch adds code that checks if the operation that we are vectorizing was vectorized during the visit of the operands and uses this value if it can. git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@186883 91177308-0d34-0410-b5e6-96231b3b80d8
* PR16628: Fix a bug in the code that merges compares.Nadav Rotem2013-07-15
| | | | | | | | Compares return i1 but they compare different types. git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@186359 91177308-0d34-0410-b5e6-96231b3b80d8
* Update Transforms tests to use CHECK-LABEL for easier debugging. No ↵Stephen Lin2013-07-14
| | | | | | | | | | | | | | | | | | | | | | | functionality change. This update was done with the following bash script: find test/Transforms -name "*.ll" | \ while read NAME; do echo "$NAME" if ! grep -q "^; *RUN: *llc" $NAME; then TEMP=`mktemp -t temp` cp $NAME $TEMP sed -n "s/^define [^@]*@\([A-Za-z0-9_]*\)(.*$/\1/p" < $NAME | \ while read FUNC; do sed -i '' "s/;\(.*\)\([A-Za-z0-9_]*\):\( *\)@$FUNC\([( ]*\)\$/;\1\2-LABEL:\3@$FUNC(/g" $TEMP done mv $TEMP $NAME fi done git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@186268 91177308-0d34-0410-b5e6-96231b3b80d8
generated by cgit v1.2.3 (git 2.25.1) at 2024-06-03 10:54:24 +0000